Just-in-time compilation

Just-in-time compilation turns frequently executed program regions into native machine code while the program is running. An interpreter or lower-level execution engine usually begins the process, collecting information such as call counts, branch behavior, and the types of values encountered. When code becomes sufficiently “hot,” the runtime sends it to a compiler and redirects later execution to the generated machine code.¹

A JIT commonly combines several levels of execution rather than compiling everything equally. Cold code may remain interpreted or use generic machine code, while hot code receives increasingly aggressive optimization. This tiered design limits startup costs and concentrates compilation effort where it can produce the greatest benefit.

The principal advantage of a JIT is that it can optimize with information unavailable to a purely ahead-of-time compiler. It may inline a frequently called function, eliminate redundant checks, specialize operations for observed types, or remove branches that have consistently behaved in one way. Java’s HotSpot virtual machine, for example, uses runtime profiling and can perform successive optimization of executing applications.¹

These optimizations are often speculative: the compiler assumes that observed conditions will continue. A guard records the assumption, and if it fails, the runtime deoptimizes the compiled code and returns to a safer version. This ability to optimize and later recover is central to dynamic-language systems and tracing JITs, whose research lineage includes Smalltalk and Self implementations.²³

JIT compilation improves long-running workloads but can make short-lived programs slower. Compilation consumes CPU time and memory, and the runtime must maintain profiling data, generated code, metadata, and mechanisms for invalidation and deoptimization. Startup latency, warm-up behavior, code-cache limits, and unpredictable workload changes therefore matter as much as peak throughput.

Modern systems address these costs through tiered compilation, background compiler threads, on-stack replacement, and code caching. A runtime may compile a quick baseline version first, then optimize it asynchronously after enough evidence accumulates. Compiler infrastructure such as LLVM’s ORC JIT APIs also supports dynamic linking, symbol resolution, and controlled compilation of code modules at runtime rather than treating JIT compilation as a single monolithic technique.⁴

JIT compilation is not limited to language virtual machines. It is used in JavaScript engines, dynamic-language implementations, database query systems, regular-expression engines, graphics and numerical libraries, and systems that generate specialized kernels. V8, for example, combines an interpreter with optimizing compilers and uses feedback from JavaScript execution to guide generated machine code.⁵

A less visible issue is that generated code can depend on the exact runtime environment: processor features, object layouts, security policies, and currently loaded libraries. Some runtimes therefore invalidate compiled code when assumptions change. WebAssembly implementations add another variation, compiling portable bytecode either during loading or progressively during execution, while balancing validation, startup speed, portability, and optimization quality.⁶

Hot code

A function, loop, or trace executed often enough to justify compilation or further optimization.

Deoptimization

The process of abandoning optimized machine code when a speculative assumption no longer holds.

Tiered compilation

A strategy that uses multiple execution and compilation levels, moving code toward more expensive optimization as evidence accumulates.

On-stack replacement

A technique that transfers an already running function or loop from interpreted or baseline execution into compiled code.