Programming by example

Programming by example is a paradigm in which a system observes a user's actions or input–output pairs and synthesizes a program that generalizes those examples. The user provides demonstrations, such as editing a few cells in a spreadsheet or formatting a string, and the system infers an underlying transformation. This contrasts with traditional programming, where the user must explicitly write instructions in a formal language.

The core challenge is the synthesis problem: finding a program consistent with the examples, which is often under-specified because many programs can fit the same data. PBE systems use search techniques, heuristics, and domain-specific languages to narrow the space. They also employ ranking functions to select the most likely intended program, often based on simplicity or prior usage patterns.¹

The concept dates back to the 1970s with systems like Pygmalion and SmallStar, which allowed users to create macros by demonstrating actions. In the 1980s, Eager and TurboTurtle explored learning from repeated behavior. However, practical adoption remained limited until the 2000s, when advances in program synthesis made PBE feasible for real-world tasks.

A landmark was Microsoft's Flash Fill, introduced in Excel 2013, which automatically fills columns based on a few examples. It was based on research by Sumit Gulwani and colleagues, who developed efficient algorithms for string transformations. Since then, PBE has expanded to data wrangling, code repair, and even end-user robot programming.

PBE is widely used in spreadsheet tools: Flash Fill in Excel, and similar features in Google Sheets and Apple Numbers. It also powers data transformation in tools like Trifacta and OpenRefine, where users demonstrate cleaning steps. In text editing, PBE can automate repetitive formatting, and in web automation, tools like CoScripter learn from user interactions.

Modern PBE integrates with large language models (LLMs) to handle more complex tasks, such as generating code from natural language and examples. For instance, GitHub Copilot and similar tools can be seen as PBE when they infer code from input–output examples. PBE is also used in accessibility, allowing users with limited motor skills to automate tasks through demonstrations.²

One niche area is programming by demonstration for robots, where users physically guide a robot arm to perform a task, and the system learns the underlying policy. Another is PBE for database queries, where users provide example rows and the system synthesizes SQL queries. These applications highlight the versatility of the paradigm.

An overlooked historical figure is David Canfield Smith, who developed Pygmalion, which introduced the concept of icons and direct manipulation. Also, the LAPIS system by Robert C. Miller allowed users to specify text patterns by example, influencing later tools. PBE has also been used in education, such as Stencils-based tools that help students learn programming by example.³

Inductive programming

A field that generates programs from incomplete specifications, such as examples.

Domain-specific language

A programming language tailored to a particular application domain, often used in PBE to constrain synthesis.

Program synthesis

The task of automatically constructing a program that satisfies a given specification.

Flash Fill

A feature in Microsoft Excel that uses PBE to automatically fill data based on examples.