Sixteen Milliseconds per Frame: Arcade, the Game Loop, and Retro Rendering

Whether an action game feels good rarely depends on its graphics; it depends on a causal chain too short to notice — press jump and the character rises the next instant, misjudge and the screen punishes you at once. This post takes the 16.67 milliseconds that chain lives in apart: first the three jobs inside one frame and measurable latency, then sixty years of loop design along two routes through Pong, Breakout and Tetris, and finally the physics behind CRT scanlines and the Game Boy’s four shades of gray. Constraints come first and style second; the point here is how constraints get absorbed by engineering and become style.

A Frame’s Budget: Timestep, Hit Detection, and Feel

Three Jobs in One Frame

Sample input, advance the simulation, draw the result. A 60Hz display gives you 16.67ms, and all three have to fit inside it.

The most common beginner version uses the real frame interval as the step size.

\text{position} \mathrel{+}= v \cdot \Delta t_{\text{real}}

It looks physical, and it is a pit. Once $\Delta t$ jitters, the same jump reaches a different height on different machines. Worse, collisions are judged on discrete positions: if one frame’s $\Delta t$ is especially large, a fast object passes straight through a wall. That is called tunneling.

The other pit is the death spiral. One frame stalls, $\Delta t$ grows, collision work gets more expensive, frames stall harder, $\Delta t$ grows again, and the tab locks up on the spot.

Engineering has had the standard answer for decades: a fixed step plus an accumulator.

1
acc += realDelta
2
while (acc >= FIXED_DT) { step(FIXED_DT); acc -= FIXED_DT }
3
render(acc / FIXED_DT)   // interpolate with the remainder

The simulation always advances at 1/60 second while the render rate moves freely. The stuttering frame is split into a few small steps and caught up, physics stays stable, feel stays consistent.

The demo above puts the two strategies side by side. On the left, fixed step: in the stalled frame the ball merely lags by a beat. On the right, real frame intervals: the ball teleports during that frame and the trajectory visibly breaks.

So the first technical fact is that logic frame rate and render frame rate are two independent things.

Feel Is Actually Measurable

Feel sounds mystical; taken apart it is just a chain. Hardware scan: 1ms for a mouse, 4-8ms for a gamepad. Browser event queue: the compositor throttles, so possibly one or two frames of waiting. Your logic frame adds up to 16.7ms more. Compositing and display wait another 16.7ms at vsync. Finally the panel responds in 5-15ms. Add it all up and one click reaching a pixel change lands between 50 and 120ms.

Humans can perceive that number.

Two things in the chain are counterintuitive.

First, the arcade’s feel advantage is essentially a by-product of hardware structure. Eighties arcade boards drove the CRT electron beam directly; there was no frame-buffer queueing at all, so input-to-pixel latency was naturally an order of magnitude lower than in a browser. Half of the so-called arcade feel comes from that physical immediacy and has little to do with design skill.

Second, tolerance for latency has a threshold, and past the threshold you can only cheat. Fighting-game players can tell about 50ms apart, while a human’s own movements carry 100-200ms of inherent error. So good games cheat in the judgment.

Coyote time: after the character steps off a ledge, jumping still works for about 100ms. The player fell long ago; the game pretends he is still at the edge. Jump buffering: presses within 100ms before landing are queued and fire the instant you touch down. Forgiving boxes: the hurtbox a size smaller than the sprite, the hitbox a size larger.

None of these three is an engine feature; they are state machines faking human tolerance. See this and you see why pixel-perfect games are often not fun: they mistake the machine’s time precision for the human’s.

A Difficulty Curve Has Only Three Parameters

Arcade packs nine modes into one entry point: flappy, jumping, frogger, dodging, traffic, flappy bird, endless runner, subway runner, platformer. The one design truth they share: the rules fit in a sentence, and difficulty rises by rhythm.

Unpack that sentence. Take the flappy family: how long a run lasts depends on three parameters — gap width $g$ , forward speed $v$ , pipe spacing $d$ . Write the per-pass success rate as $p$ ; the probability of surviving $n$ pipes is $p^n$ , exponential decay. One formula explains three design phenomena.

To lengthen a single run, widening the gap works far better than adding pipes, because $p^n$ is extremely sensitive to $p$ . To manufacture the frustration of just missing, dial $p$ to around 0.9: the first ten pipes pass 35% of the time, the first twenty only 12%, and the player keeps chewing over “I could have made it”. Escalating difficulty needs no new mechanic either — just let $p$ drift down over time.

Engineering-wise there are only three levers: speed up, densify, hide information. Crossing the river uses all three — lane speed climbs, safe cells are squeezed away one by one, and the logs in the river start shrinking.

One detail usually goes unnoticed. In 1978’s Space Invaders the difficulty curve was that the fewer enemies remained, the faster they stepped down; that was not a designer’s careful arrangement but a by-product of the hardware budget. That 8080 could move only one enemy per frame, so the more enemies were destroyed, the faster each polling round, and the more often the whole formation stepped down. Many classic arcade difficulty curves were originally artifacts of insufficient compute, read by players as design.

Collision and Budget

Hit detection is the physics engine of every action game, and it chooses between only two geometries: an axis-aligned bounding box or a circle. AABB is four comparisons; a circle is two multiplications and one comparison — cheap enough to sweep hundreds of objects per frame. The price is precision: a bullet flying diagonally grazes air under AABB. Hence swept tests, treating this frame’s displacement as a segment and intersecting it with the target’s rectangle — the specific cure for fast-object tunneling.

What actually eats the budget is the pair count. In a Shoot ‘Em Up bullet-hell scene with 200 bullets, 100 enemies and 1 player on screen, brute-force pairwise testing is $\binom{301}{2}$ , about 45,000 checks per frame. JS alone can run that; add particles and rendering and it gets dicey. The standard optimization is spatial partitioning: cut the screen into a grid and test only the same and neighboring cells, dropping complexity from $O(n^2)$ to nearly $O(n)$ . Half the performance budget of those dense-looking bullet-hell screens goes here.

One more thing has to be kept separate: visuals and judgment. The sprite is for people, the hitbox is for the machine, and the two need not coincide. Shrink the hurtbox by 20% and difficulty drops 20% with nothing visible on screen. This is also the shared secret of so-called hardcore games: hard in the numbers, not in the judgment.

How Limits Become Style

Game Boy Filter does something plain: map every pixel of an arbitrary image by brightness onto the four green-grey levels of the DMG screen. The principle is nearest-neighbor quantization — compute luminance, find the nearest of the four levels, replace — with no dithering anywhere. The result is that gradients get chopped into flat blocks of solid color and detail is brutally flattened.

That was exactly the situation in 1989: 160×144, 2-bit color depth, no backlight, relying on ambient reflection, response time measured in hundreds of milliseconds. Artists had no choice but to subtract within four grays — separate layers with brightness steps, patch shapes with outline strokes, hint at material with a single-pixel highlight. The constraint came first, the style second; style is the constraint’s solution.

The CRT effect above runs the same idea in reverse. Scanlines, phosphor afterglow, screen curvature — all of that retro feel comes from the display device’s physical defects. Retro aesthetics can be summed up in one sentence: turn the previous generation’s hardware error into this generation’s design language.

Rhythm Games Don’t Trust Timers

Rhythm Game collects Piano Tiles, Taiko and Rhythm Master, and it exposes time-handling mistakes better than anything else.

The intuitive version generates notes with setInterval(1000 * 60 / bpm). In a browser this fails for certain, for three stacked reasons. Timer resolution is about 4ms, and the spec even allows throttling in background tabs to a second or worse. Error accumulates on every callback. Audio output has its own buffer and does not sync with DOM timers. At BPM 120 a sixteenth note sits 125ms apart, and 10ms of jitter is enough for the player to hear a stumble.

The right approach makes the audio clock the only timebase. Take AudioContext.currentTime as truth, schedule notes into the audio queue with a few milliseconds of lookahead, and let the visuals be nothing but a projection of the audio timeline. That is what the site does: convert one beat with 60000 / bpm, compute each note as an absolute time point, schedule the oscillator at $t_{\text{start}} + t_n$ , and judge by the distance between the note position and the judgment line. Audio is the master clock, the picture is the slave; invert that relationship and you never land on the beat again.

While we are here, three tools that measure you with games. Reflex Games measures simple reaction time: the human floor is about 200ms, normal people sit at 250-300ms, and scores depend on refresh rate and input device, so cross-device comparison means nothing. Typing Speed Test uses the WPM algorithm, five characters to a word, $\text{WPM} = \frac{\text{characters}/5}{\text{minutes}}$ ; the site updates WPM live and computes accuracy character by character. Note that net WPM has to subtract error characters — many tests report gross WPM and systematically overestimate. Memory Match measures working memory: the current estimate is 4±1 chunks, beyond which you necessarily fall back on rehearsal strategies rather than retention.

As for training value, Arcade trains the speed of the loop between perception and action — narrow transfer, quick results. Rhythm Game trains temporal resolution, with positive transfer to music, movement and speech rhythm. 2D Simulation goes the opposite way entirely: it does not want instant feedback, it wants to know how a deterministic system grows unpredictable. The simple pendulum’s harmonic motion, the double pendulum’s chaos, the Lorenz attractor, N-body gravity — seen together they form exactly the negative of an action game. The same $F=ma$ becomes feel when made realtime, and chaos when taken offline.

Sixty Years of the Game Loop: From Pong to Tetris

The fixed step plus accumulator from the previous chapter is no fresh invention; it is the endpoint of a loop skeleton iterated over sixty years. Strip any real-time game down to its skeleton and the same structure remains: read input, update state, draw a frame, go back to the start — true of the vector-display games in 1962 labs and of Canvas games in today’s browser alike; only the internals of each stage change.

The skeleton of a loop

Prehistory of the loop

First, settle where the “sixty years” start counting. The oscilloscope tennis at Brookhaven in 1958 and MIT’s PDP-1 in 1962 produced the first real software game loops: Spacewar!‘s code ran around the refresh cadence of a vector display — read the console switches, compute two orbits, draw two ships — and the three-stage shape was already complete. Pong in 1972 carried that skeleton into the arcade and became the first real-time game loop the public ever saw, and for a decade after, the three-stage structure did not change in essence.

A game loop is the product of one root constraint: a program serving a screen that refreshes periodically. As many years as screens have refreshed, the loop has that many years. The oldest game loop is a bare while(true): read input, advance state, draw, as fast as it will turn. Arcade machines had no operating system, so the loop’s cadence came from the hardware clock and the CRT’s electron beam scanning; rendering meant “emit the right signal at the instant the beam passes”. The modern browser equivalent is requestAnimationFrame (rAF): it hangs your callback on the display’s refresh cadence — a 60Hz screen calls back every 16.7 ms, a 144Hz screen every 6.9 ms — and you hand control back after updating and drawing. Compared with timer polling, rAF has two decisive advantages: it aligns with vertical sync, so no half-frames get drawn; and it stops automatically when the page is hidden, so background tabs do not burn CPU.

Each of the three stages has its own discipline. Read input: either event-driven (keys queued, consumed during update) or a polled snapshot (current key state read every frame). Update advances all game state; input latency is mostly decided by how input is queued in this stage. Render should be a pure function: read state, never write it — that is what makes pause, replay, and swapping render backends possible. The stage easiest to get wrong is update, because it drags in the oldest problem in game programming: the timestep.

Variable and fixed timestep

Tunneling and the death spiral got named in the previous chapter, and a variable timestep hides a third hazard: because dt differs per frame, physics results drift slightly with frame rate — the same input can trace slightly different paths at 60Hz and 144Hz, which racing and fighting games tolerate at zero.

The standard engineering fix is the full version of the previous chapter’s skeleton, the well-known “Fix Your Timestep” pattern: the simulation runs on a fixed clock, the renderer runs on the real clock, and the accumulator is the currency exchange between the two. Physics advances in a fixed step (1/120 s, say); each frame adds real elapsed time to a buffer, and while the buffer holds a full step, advance one step and subtract it; rendering always uses the latest state, with interpolation between the two most recent physics states when more smoothness is wanted.

1
let acc = 0, last = performance.now();
2
const STEP = 1 / 120;
3
function frame(now) {
4
  acc += Math.min((now - last) / 1000, 0.25); // cap to prevent the "death spiral"
5
  last = now;
6
  while (acc >= STEP) { update(STEP); acc -= STEP; }
7
  render(acc / STEP);                          // interpolation factor
8
  requestAnimationFrame(frame);
9
}

The Math.min line deserves its own explanation. If one frame stalls for 5 seconds, the accumulator collects 5 seconds and the loop runs six hundred physics updates back to back; the worse the machine stutters, the more it accumulates — a vicious circle, the “death spiral”. Cap the per-frame catch-up and, on a hitch, you would rather the game look like slow motion.

Background tabs are a browser-specific pitfall. rAF stops, timers are clamped to once per second, and on return the accumulator receives one large lump of time. Games with pause semantics should simply pause there, which is the most dignified handling; games without them should at least let this cap absorb the lump.

Pong

Strictly speaking, Pong in 1972 had no software loop. It was a slab of TTL logic, with position counters generating ball and paddle signals straight against the TV raster, and the “game logic” was a handful of comparators: first to 11 resets the score.

But translated into a modern software loop it is the highest-value teaching starting point, because its state fits in five numbers: the y coordinates of the two paddles, the ball’s position and velocity vector, and both scores.

The update rules are a few lines per frame: the ball flips its vertical component when it hits the top or bottom edge, flips its horizontal component when it hits a paddle, and scores and reserves when it leaves the field.

Pong’s most important design legacy is that the bounce angle is decided by the hit position. A ball striking the paddle’s center bounces at a shallow angle, at the edge at a steep one. One linear mapping hands “player control” to you for free, with no physics engine at all.

Now the AI opponent. Move the computer paddle toward the ball’s y coordinate, but put a cap on its speed, and the difficulty hides in that cap. Slow paddles miss fast balls; fast ones are nearly unbeatable.

The site’s Pong is exactly this structure. Classic mode controls the paddle with the arrow keys (or WASD, or the mouse), while the computer paddle tracks the ball’s y with delay and error. The AI-versus mode offers three difficulty tiers, and the tier maps to one of three AI paddle speed caps.

Another benefit of a minimal loop is determinism. Every state transition is decided by input; there are no random numbers and no chained physics sensitive to floating-point error. Loops you can exhaust in your head are the datum plane for understanding all more complex loops.

The circuit era also left software one hidden inheritance: rendering cadence must serve the scan.

Pong’s signal generation was locked hard to the TV raster, and missing a line meant a torn picture. Today’s rAF-to-vsync alignment and the V-Sync switch in a game engine are the same constraint echoing through a modern pipeline.

Put the sentence “decouple update from display, but align the update cadence with the display cadence” into a 1972 cabinet and it holds just the same.

Breakout

Breakout in 1976 added exactly one thing to Pong’s skeleton: a wall of bricks. The loop’s complexity, though, rose a notch.

Bricks are a textbook entity array. Each brick carries coordinates, size, and a live flag, and collision detection each frame becomes “run an AABB test between the ball and every live brick”; a hit flips the ball’s velocity component, marks the brick dead, and scores.

Once state multiplies, three things follow.

First, clear detection. When the live count hits zero the next level begins, and the level change resets ball position and speed. In the site’s Breakout the ball speeds up linearly per level, and the three-lives configuration is a convention inherited from the arcade original.

Second, tunneling. With a fast ball and thin bricks, one variable-timestep step can step right past a brick: the collision test misses at both ends and the brick is “passed through”. The cheap fix caps ball speed or splits a frame into substeps; the expensive one is continuous collision detection (sweeping the trajectory). Pixel games almost always take the former.

Third, state ownership. Ball, paddle, bricks, lives, level, score — who owns each, and on a restart which get reset and which survive. Breakout’s code size already needs a little architectural awareness, but the loop skeleton is still read input, update, render.

The arcade original has two more easily missed economic designs.

Combo acceleration: after the ball hits paddle and bricks some number of times in a row, everything speeds up, closing off the “play it safe” route.

Position scoring: higher rows score more, encouraging the player to take risks and drive into the wall.

That is already the germ of a difficulty curve: use score to lure the player into faster ball speeds.

The three lives are the outer layer of the same economics. A life is a trial-and-error budget, smoothing the cost of a mistake from “game over” into “one life lost”. Until the budget runs out the player stays on the climbing part of the learning curve. Every arcade coin mechanism that followed is a commercial reissue of this budget system.

Tetris

In 1984, Pajitnov at the Soviet Academy of Sciences wrote Tetris on an Elektronika 60 with no graphics capability. The blocks were assembled from text characters, and the gameplay was complete from the start: seven tetrominoes fall into a ten-wide well, and a full row clears.

Unlike Pong and Breakout, this is a new loop route altogether. The game world is a discrete grid, pieces align to cells, and there is no continuous displacement, no inertia, no tunneling. Collision is a lookup: “is the next cell occupied”.

The loop’s shape changes with it. Gravity is no longer “displace some pixels per frame” but a state transition fired on a timer. The active piece goes from falling to locked, a new piece spawns, full rows clear, repeat.

The logic beat and the render frame rate are thereby decoupled. Pieces drop one cell per second while the picture interpolates at the display’s refresh rate; two clocks, each running its own way.

That is also why the site’s Tetris uses a timer rather than a frame counter for gravity. Every 10 rows cleared is a level up, and the drop interval shortens per level with a floor. Put the beat on a clock and changing difficulty is just changing one interval number, entirely independent of render frame rate.

This choice has a browser-specific side effect. Background-tab timers are clamped to once per second, so switching away and back actually makes the pieces “auto slow-motion”.

Discretization also brings a new input design problem.

In a continuous game, holding a direction key is a per-frame poll; in Tetris the counterpart is DAS (delayed auto shift). Press and the piece moves one cell, then after an initial delay it goes into fast repeat. Without it, pushing a piece to the wall would take twenty-odd presses.

The “modern guideline” settled in the 2000s filled in the rest of the state machine design.

Lock delay: after a piece lands it gets a fraction of a second before locking, and movement can refresh it a limited number of times.

Soft drop and hard drop: one key slams the piece to the bottom and locks it instantly.

The 7-bag randomizer: the seven pieces are shuffled into a bag and dealt in order before the next bag is shuffled, eliminating the despair of four S pieces in a row.

All of these are modifications of state transition rules, without touching a line of physics code. In a discrete game, “feel” lives almost entirely in the state machine, not in physics parameters. The contrast with continuous games is beautiful: tuning feel in Pong means tuning paddle speed and bounce angle; tuning feel in Tetris means tuning timing parameters.

Why interpolation is necessary

Under a fixed timestep, beginners often skip the render interpolation step, and the symptom of skipping it is a very particular “micro-jitter”.

Unpack the mechanism: physics advances every 8.33 ms (1/120 s) while the display wants a frame every 16.7 ms. The two cadences are like two watches running at different rates; on some frames physics has just taken two steps, on others two and a half, so the object’s screen position jumps between “where it should be” and “half a physics step further”.

Sixty tiny jumps a second read to the eye as high-frequency shimmer, especially on fast-moving objects.

The fix is to pass the 0-to-1 coefficient acc / STEP into rendering and let it linearly blend the object’s two adjacent physics states. The drawn position is not “the most recent physics position” but “the position that should be on screen at this fraction of time”.

The cost is that every rendered entity keeps its previous state: memory traded for smoothness.

Skipping interpolation can survive too: set the physics step equal to or a divisor of the refresh rate and the jitter disappears. But multi-monitor compatibility collapses immediately, so serious implementations interpolate.

Input latency

The previous chapter measured 50-120ms end to end; the floor of that mystical metric, responsiveness, hides in every link of the chain. A single keypress in a browser runs this pipeline: keyboard hardware scan, OS keyboard buffer, the browser main thread dispatching the event to the page, the page recording the key into game input state, the next frame’s update consuming it, the render into the canvas, and finally the compositor putting the new frame on screen. Seven or eight pipeline stages, each possibly a few to a dozen milliseconds.

For most games this is imperceptible; for fighting games and rhythm games it is life or death.

A few things are within the page’s control:

Mixing events and polling. Text input uses events (no dropped keys); directional input polls a snapshot (events do not report the held state). Most game loops use both.

Coalesced events. High-frequency pointer events are merged into one delivery between frames, so when you need per-point trajectories you use the interface that reads coalesced events and returns the intermediate points.

Record input first and consume it in update. Mutating game state directly in an event callback bypasses the discipline that update is the only state entry point. Then keys leak into the game while paused, and replay becomes impossible.

Networked games push latency management to the extreme. Client-side prediction and rollback sync (run locally first, correct after confirmation) is the ceiling of latency engineering, and that is another article’s worth of material.

The two routes converge

Put the three games and the site’s implementation choices side by side and the conclusion surfaces on its own.

Pong and Breakout are continuous physics: state evolves continuously with time, and an rAF frame loop with velocity integration is the most natural fit.

Tetris is a discrete state machine: state jumps only at event instants, so a clock-driven beat with rendering as a separate layer is the most natural fit.

In code both routes land on the same skeleton: a fixed-step update, a state-reading render, and rAF or a timer as the heartbeat. The only difference is whether update integrates or looks up.

There is also a shared periphery.

Canvas device-pixel scaling: multiply the canvas size by the device pixel ratio and scale back, or the whole game looks blurry on a high-DPI screen.

Pause on window blur.

Map touch zones to input state on mobile.

These three small things genuinely decide whether the game is playable at all, and they outrank any algorithm.

Randomizers and determinism

Only Tetris among the three cannot do without random numbers (the 7-bag), and behind random numbers hides a deeper principle of loop design: randomness must converge onto a deterministic seed.

The engineering practice is to write a lightweight PRNG (a dozen lines, xorshift and the like), seed it once with the current timestamp, and let every subsequent “random” number be a deterministic function of the seed.

The payoff comes in two places.

First, reproducibility. When a bug appears, rerun with the same seed and ten thousand games take the same track; tuning and repro do not depend on luck.

Second, replayability. Record “the seed plus all input” and the whole game replays from the beginning. Stream replays, anti-cheat audits, and networked sync (both sides exchange only the seed and input, never state) all use this trick.

The browser’s built-in random function is convenient, but it cannot hand back its seed, and that kills reproducibility.

One more guard against sloppiness: when mapping a float to an integer range, avoid a bare modulo (low bits are unevenly distributed) and scale by multiplication instead. floor(rand() * n), not rand() % n.

Each detail looks small on its own; together they decide whether the game’s loop is testable.

Mobile touch

For a browser game to be playable on a phone, there are three compulsory questions outside the loop.

Suppress default gestures. Swiping on a canvas scrolls the page along with it, and one touch-action: none declaration hands the touch to the game.

Virtual control layout. Left, right, rotate, and soft drop demand at least four touch zones, with the thumb hotspots placed for one-handed grip and hit targets at least 44 pixels square. Touch events likewise must be recorded into input state and consumed by update, treated exactly like the keyboard. Two input paths feeding one state layer is what prevents asymmetries like “touch players bypass DAS”.

Performance tiers. Phone GPUs are sensitive to full-screen redraws every frame, so skipping redraw on static scenes and adapting particle counts to the device are the usual ways to hold frame rate.

Mobile is not the desktop build shrunk a notch; it is the input layer reimplemented. The loop skeleton stays, the input adapter is new.

Three minimal checks

Before building your own loop, pass three minimal checks.

Frame-rate independence. Move the display from 60Hz to 144Hz, or throttle the CPU several times over in devtools; the same input should trace the same path. If the path changes, some physical quantity is still hidden in “per frame” instead of “per second” — hunt each one down and convert it to a time unit.

Pause integrity. Hammer the arrow keys while paused, then resume; state should be identical to before the pause, bit for bit. This verifies the discipline that input is consumed only in update.

Tab-switch return. Leave for minutes and come back with no tunneling, no teleporting, no double speed. This verifies the accumulator cap and the blur handling.

Pass all three and the loop skeleton stands.

The Physics of Retro Signals and Pixel Rendering

Where the loop tames time, the rest of the budget goes to pixels. Old display devices were riddled with defects: scanlines sliced the picture into horizontal stripes, phosphor trailed a lingering afterglow, the curved glass pressed the corners dark, and magnetic tape tore colors into frayed edges — things engineers of the day filed as defects to be fixed got reverse-canonized into an aesthetic in the digital era, because they carry a material texture unique to analog signals that modern screens, lighting up point by point, will never hand over on their own. Recreating them needs neither WebGL nor a shelf of shaders; Canvas 2D plus a few DOM layers is enough, and the real difficulty is working out the physical cause of each defect, then translating it into a matching code structure.

Why Flaws Became an Aesthetic

First, be clear about what a CRT actually does.

An electron gun fires a beam, deflection coils use a magnetic field to sweep it across the phosphor coating left to right and top to bottom, line by line — the process is called raster scanning. Struck by electrons, the phosphor glows and then decays along its own afterglow curve within a few to a few dozen milliseconds.

Strictly speaking, at any instant a CRT screen holds only one moving bright line; the rest of the picture is a “gradually fading memory” held up by phosphor afterglow. It is a device that plays back line by line, not a plane that exists all at once.

A color tube adds a shadow mask or an aperture grille between the beam and the coating, so the beam only strikes phosphor dots of the matching color.

So a CRT’s “pixels” were never a tidy grid of glowing points but a physical coating of three phosphor groups, red, green, and blue. Boundaries are soft, and at high brightness they bleed light into the neighborhood.

Add a 50/60 Hz refresh rate, the curvature of the glass envelope, and the illumination falloff from thicker glass at the edges, and that is the whole list of an old monitor’s “illnesses”: scanlines, afterglow, flicker, vignetting, geometric distortion.

The broadcast era stacked one more layer of blur along the time axis. Analog television used interlacing: each frame was split into odd and even fields sent alternately, trading field rate for full frame rate, and static images were filled in by afterglow and persistence of vision into a complete frame.

Old footage looks “smooth” half because of spatial phosphor bleed, half because of this temporal blur.

Most recreation filters today restore the spatial dimension only. Scanlines, vignetting, and chroma aberration are all easy; afterglow and interlacing need per-frame motion estimation, with cost and benefit wildly out of proportion, so nearly every web implementation (including the ones in this article) steers around them.

Those defects were elevated to an aesthetic because the game art of the 8-bit and 16-bit era was designed around them in the first place.

At low resolution, large dither checkerboards were smeared by the CRT’s glow into smooth transitional color. Scanlines gave the picture a clear “row structure” that made 320×240 content look sharper than it was. Phosphor bleed served as natural anti-aliasing, softening the stair-steps at the edges of color blocks.

Put the same picture on a point-by-point LCD and the dither reverts to a noise pattern while the jaggies show themselves again. The defects are gone, and the design intent of the day fails with them.

So the essence of a retro filter is not “adding noise to the picture” but putting a layer of optical imprecision back into an exact digital display.

Optical Layer Stacking

There are two fundamental routes to recreation. The first is optical layer stacking: the picture underneath stays untouched, and each defect becomes a transparent overlay.

The site’s CRT Filter is a complete specimen of this route.

By default it draws a demo scene: a grid of 22px cells on dark green, 24 bright spots flickering along pseudo-random trajectories, a ball bouncing along a Lissajous path, the title FROSTI, and a PRESS START blinking every half second. Upload a local image and any picture replaces it.

That demo scene is not drawn casually. A regular grid is the test pattern most likely to expose scanlines swallowing rows and geometric warping, and the high-contrast ball checks how flicker reads in motion. Old TV repairmen ran the same test with checkerboard signals.

On top of the picture sit four independently toggleable overlay layers, plus an intensity slider from 10% to 100%.

Scanlines: a CSS gradient. A 1px-tall dark line every 3 CSS pixels, opacity $\alpha_{scan} = 0.15 + 0.35 \times I$ ( $I$ is the intensity). A scanline is, at bottom, spatial square-wave sampling, and 3px is an empirical balance: dense enough for the eye to merge the dark lines into a “row structure”, loose enough not to interrupt the picture.
Flicker: a full-screen white layer whose opacity is driven by a CSS keyframe animation named crt-flicker, amplitude $\alpha_{flick} = 0.12 \times I$ . The keyframes have only two frames and the timing function is steps(2); opacity hard-switches between the two levels $0.5 \pm \alpha$ , so the flicker is a square wave rather than a breath. Effects like this, pure presentation, run on the compositor thread and never touch the main thread for a single frame — never drive styles from JS per frame.
Vignette: a radial gradient, fully transparent over the central 55% and fading to 65% black at the edges. It matches the illumination falloff from thicker glass and a steeper incidence angle at the tube’s rim; photography’s cos⁴ law says the same thing.
Curvature: 28px rounded corners on the container, then a perspective(900px) rotateX(2deg) 3D transform and an inner shadow on top — the cheapest approximation of a convex glass envelope.

The intensity slider drives two derived quantities at once, scanline opacity and flicker amplitude; in the source both hang off the same reactive state, so turning the slider makes the two defects rise and fall together. Semantically it is one “total damage level”.

One engineering detail deserves a mention. The demo scene repaints every frame inside a requestAnimationFrame loop, but all four overlays are static CSS. Toggling an effect triggers no repaint at all, it only shows or hides a DOM layer.

“Zero-cost effect toggling” is not rhetoric; it is structure.

The engineering takeaway of the whole route is therefore one sentence: the effects are all overlay layers, and the underlying pixel data is never touched, so the layers cannot contaminate each other.

The Scanline Effect goes one step further along this route and draws the shadow mask as well.

It has two layers: the scanline layer, isomorphic to the CRT filter’s ( $\alpha = 0.1 + 0.5I$ ), and the crucial RGB mask layer — a horizontally repeating gradient with red, green, and blue each taking 1px in a tiled cycle, overlaid at opacity $\alpha = 0.15 + 0.5I$ in screen blend mode.

This imitates the color-separation structure of a color picture tube: vertical stripes of the three phosphor colors (the aperture grille of the Trinitron school).

Both layers have a 3px period, but one runs vertically and the other horizontally; the orthogonal rhythms don’t fight, which is why the stack shows no unpleasant interference patterns.

Why must the blend mode be screen? Because phosphor is additive light. The screen formula is:

C_{out} = 1 - (1 - C_a)(1 - C_b)

The two inputs each contribute brightness independently; things can only get brighter, never darker. That is light upon light.

With multiply, the model of a subtractive pigment system, the picture would darken fast. With ordinary alpha compositing, the mask would simply blot out the base image.

Behind the choice of one blend mode sits the divide between two color systems.

Signal Layer Rewriting

The second route is signal layer rewriting: the effect cannot come from stacking, the picture must be rewritten pixel by pixel.

The VHS Filter is the representative of this route, because the defects it recreates live in the signal, not in the optics.

VHS is analog tape recording. The heads sweep the tape surface in a helix, the luma signal is recorded directly, and the chroma signal is down-converted and squeezed into the low end of the spectrum (color under).

On playback, tiny jitter in tape speed and misalignment between head and track turn into defects visible to the naked eye.

The component maps each defect to a slider; the defaults and the prescriptions are as follows:

Cause	Recreation method	Default
Tape jitter, uneven tension	Horizontal offset per row, wobbling along a sine	Distortion 0.5
Chroma playback misregistration	Three channels sampled at horizontal offsets	Chroma 4px
Head-switching noise	A rolling horizontal band of noise	Tracking 0.6
Weak signal, high noise floor	Additive grayscale noise	Noise 0.4
Row structure	A 1px dark line every 3 rows	Scanlines 0.5

The core is a per-pixel loop.

The source image (the default scene is 26 random semi-transparent circles on a gradient with a line of ▶ PLAY, like a frame paused on a tape; an upload works too) is first kept as an ImageData base copy, then each frame produces the output:

1
for (let y = 0; y < sh; y++) {
2
  // tape warp: row index as phase, the time term rolls the wave along
3
  const warp = Math.round(Math.sin(y * 0.25 + time * 4) * distortion * 10);
4
  for (let x = 0; x < sw; x++) {
5
    const sx = ((x - warp) % sw + sw) % sw;
6
    const i = (y * sw + x) * 4;
7
    od[i]     = sd[(y * sw + ((sx + shift) % sw)) * 4];      // R shifted right
8
    od[i + 1] = sd[(y * sw +  sx) * 4 + 1];                  // G in place
9
    od[i + 2] = sd[(y * sw + ((sx - shift) % sw)) * 4 + 2];  // B shifted left
10
  }
11
}

The chroma step samples the red channel shift pixels to the right and the blue channel to the left, leaving green in place. Color fringing from poor chroma/luma registration is translated into offset reads of three channels.

The tracking bar is a band $h \times (0.06 + 0.12 \times T)$ tall, its position rolling up and down the frame along a sine; every pixel inside gets ± $60T$ of noise plus 30 on the red channel.

So it is that pinkish bright streak crawling upward — exactly what head-switching noise looks like on an old TV.

The noise adds the same random offset to all three channels of every pixel; the shared value guarantees grayscale noise rather than colored speckle.

Two implementation details.

One: the wrap-around after the row shift uses ((x - warp) % sw + sw) % sw, adding the width before taking the modulus. JS’s % keeps the sign of a negative number, so a plain modulo would roll negative indices out of the left end of the row; add sw once and the picture truly connects end to end.

Two: the noise is re-randomized every frame. The spatial pattern changes every 16ms, which is exactly where the “aliveness” of an analog noise floor comes from. A static noise pattern looks fake at a glance; only a moving one reads as signal.

The performance bill deserves a look too. For a 560px-wide source, each frame recomputes O(W×H), some two hundred thousand iterations, with only modulo and array reads inside; JavaScript still holds 60fps.

That magnitude is the ceiling of the per-pixel route. One order of magnitude higher and it is time for WebGL.

Incidentally, ImageData is backed by a Uint8ClampedArray, which clamps writes to 0–255 automatically and saves every handwritten clamp. For image processing in the browser, typed arrays are the first cornerstone (for the language-level groundwork, see the early JavaScript notes).

The “Reset” button hides a teaching point too. It twists the five sliders back to their defaults: distortion 0.5, chroma 4px, scanlines 0.5, noise 0.4, tracking 0.6. That default set is itself a “typical VHS case file”.

Pull all the sliders to zero and the picture snaps back to a clean digital image. In the per-pixel route disease and remedy correspond one to one: switch off each disease and it hands you back one kind of cleanliness.

The Static Screen

The TV Static recreates the most abstract defect of all: no signal.

When the antenna receives nothing, the TV’s automatic gain control (AGC) pushes the amplifier gain to maximum, and resistor thermal noise, atmospheric noise, and electromagnetic leakage from nearby equipment all get amplified into visibility.

Static is not “some pattern” but a noise floor amplified into view.

White noise is defined mathematically as a flat power spectrum; on screen that means every pixel is independent and identically distributed.

The component’s implementation is not a uniform distribution, though. Every frame, every pixel independently draws once from a two-point mixture:

with probability $1 - I$ it takes dark noise: $v = 10 + 16\,u_1$ , near black but not pure black;
with probability $I$ it takes a bright pulse: $v = 110 + 145\,u_2$ .

All three channels share the value, pure grayscale.

The intensity slider (5%–100%, default 60%) moves exactly the bright-pulse probability. The source comment puts it plainly: the higher the intensity, the more bright speckle.

Why not a uniform distribution? Because uniform white noise reads as an even layer of gray fog, whereas the memory of “static” is sparse bright points jumping over a large dark field — a bimodal luminance histogram.

This is the shared lesson of every noise recreation: the shape of the distribution decides the look, and a uniform distribution is almost never the answer.

The slider’s range hides one more nicety: the intensity floor is 5%, not 0. A fully black screen is not “the weakest static”; that is the power button’s jurisdiction, and the two kinds of “no content” are physically different things.

Three layers are permanently stacked on the picture: a 1px scanline every 3 rows (opacity 0.28), a white flicker layer with a random per-frame strength ( $\alpha = 0.12(1-I)\,u$ — the higher the intensity the weaker the flicker, since the bright speckle already flickers enough), and a radial vignette.

Power the set off and the screen settles to a near-black #080808. An old TV’s switched-off screen is the gray-black of undispersed afterglow, not an OLED’s absolute black — this detail is covered too.

This noise has an exact physical provenance: the root-mean-square voltage of resistor thermal noise:

v_n = \sqrt{4\,k\,T\,R\,B}

where $k$ is the Boltzmann constant, $T$ the absolute temperature, $R$ the resistance, and $B$ the bandwidth.

Every one of those electronic disturbances in the antenna feed, once the AGC has the gain maxed out, is amplified into a flicker of light and dark on the screen.

Static is the second law of thermodynamics signing its name on glass, and that sentence holds literally.

Incidentally, that also made it the cheapest hardware random number generator in history: thermal noise is a ready-made entropy source. This component goes the other way, feeding Math.random into a “fake TV”.

Game Boy’s Four Shades of Gray

The 1989 Game Boy (DMG-01) had only 160×144 resolution, four shades of gray, and a non-backlit reflective LCD.

Painting light and shadow with four colors forced out an entire aesthetic of constraint: dithering for transitions, heavy outlines for contrast, negative space as highlight.

The Game Boy Filter recreates exactly this palette rendering under constraint.

The pipeline has four steps.

First, the uploaded or dragged-in image is scaled proportionally to a maximum width of 220px (the built-in sample is 220×160, the same order as the handheld’s resolution, and this step doubles as downsampling anti-aliasing).

Second, compute luminance per pixel:

L = \frac{0.2126\,R + 0.7152\,G + 0.0722\,B}{255}

These are the ITU-R BT.709 HDTV weights; the eye’s sensitivity to green is baked into the coefficients.

Third, nearest-neighbor quantization against the luminance of the four palette entries.

The four colors are #0f380f, #306230, #8bac0f, #9bbc0f. This classic “dark green to yellow-green” palette works out to luminances of roughly 0.17, 0.33, 0.60, 0.66 under the formula above.

Note that they are not evenly spaced: the two brightest differ by only 0.05, nearly touching, so the highlight range is effectively folded into one level.

That is where the recreation is faithful. The DMG’s LCD response compressed highlights to begin with and the picture leans mid-to-dark, so applying the palette carries a built-in “pocket device” muffled feel.

Fourth, no dithering at all: nearest-neighbor keeps the hard edges of color blocks. Before and after are shown on two canvases side by side, alongside a toggleable scanline layer (1px every 3 rows, opacity 0.25) and a four-swatch palette strip at the bottom (hex values labeled).

The component doesn’t just hand you the result, it puts the palette itself on display — exactly what a textbook on the aesthetics of constraint should look like.

The quantization math earns a note. 4 levels = 2 bpp; for uniform luminance input, nearest-neighbor quantization has a maximum error of half the gap between adjacent levels and an average error of about a quarter of the gap.

For something smoother there are only two roads: add levels, or dither, i.e. trade spatial resolution for luminance resolution.

Classic GB games took ordered dithering: tile a 4×4 Bayer threshold matrix across the screen and let each pixel decide from the threshold at its own position whether to “step up a level or grit through it”, which from a distance reads as a continuous gradient.

This component deliberately skips dithering, leaving the quantization boundaries clearly visible, which actually reads as more “pixel art”. The constraint is preserved as style.

There is also a leave-the-mistake-alone choice worth recording. The whole luminance computation and quantization run directly on sRGB (gamma-encoded) components. A strict imaging textbook would demand gamma expansion first, averaging in linear light, and re-encoding at the end.

But the reference object of a retro filter is itself an old LCD with a nonlinear response; applying the full linearization pipeline would come off as “too correct”.

In engineering judgment, knowing where you can be imprecise matters more than being precise everywhere.

One more amusing contrast: the site’s ASCII Renderer computes luminance with 0.299/0.587/0.114 — the BT.601 standard-definition weights.

Both are international-standard approximations; neither is wrong, they just cite a television from a different decade. Two components on one site each using one set makes for a live elective course in colorimetry.

From Gray Levels to Characters

Character art has an older pedigree than pixel art.

Line printers had no pixels; a printer could only print characters, so people assembled images from the ink density of characters.

The principle in one sentence: treat each character as a “super-pixel”, with the ramp ordered monotonically by ink coverage. A space covers 0%, a period barely touches, @ is nearly filled in.

Under the terminal habit of white on black, bright regions map to @ and dark regions fall to spaces.

The ramp the ASCII Renderer uses is exactly these ten characters: ” .:-=+*#%@”.

Its pipeline is another variant of gray-level quantization.

The source canvas is fixed at 320px wide (the default scene is a radial gradient with five black circles and two white bars, or an uploaded image). Horizontal resolution is adjustable between 40 and 160 columns (default 90), cell width = 320 ÷ columns, cell height = width × 2.

A monospace glyph is roughly twice as tall as it is wide; without the factor the picture gets squashed vertically. This is the most classic pitfall of character rendering.

At the default 90 columns over a 320px source, each cell is about 3.6px wide and 7px tall. At that granularity the characters still resolve as shapes and the picture still resolves as structure — the sweet spot where both ends stay legible.

Each cell is sampled at a step of 2 and averaged (a box-filter downsample), with BT.601 weights.

Then a density parameter (0.4–2.4, default 1.2) stretches the contrast:

L' = (L - 128) \times D + 128

$D < 1$ compresses the gray scale and gentles the character transitions; $D > 1$ spreads the contrast and makes light and dark more decisive.

After stretching it is clamped to [0, 255], and the character is looked up at $idx = \lfloor L'/255 \times 9 \rfloor$ .

The ramp itself is ordered by hand. The same character has different ink coverage in different fonts, so ” .:-=+*#%@” is an approximate ranking of perceived density rather than a precise scale.

But the ordering is unshakeable, because in any monospace font @ is denser than a period.

Abstract the pipeline once more: averaging luminance per cell is a low-pass filter, there to suppress the aliasing downsampling inevitably brings. Skip the averaging, sample directly, and high-frequency detail comes back as false moiré.

The result is a piece of plain text in a monospace block. What you copy away is characters, not a bitmap — the “copyability” unique to the ASCII route.

Game Boy 4 levels = 2 bpp, ASCII 10 levels ≈ 3.3 bpp, 8-bit gray 256 levels.

The fewer the levels the harder the edges, so all retro rendering slides along one exchange curve: spatial resolution ↔ luminance resolution.

Push the columns to 160 and each character gets a 2px sampling face: detail returns, the character feel fades. Squeeze to 40 and only silhouette and volume are left.

That, together with the Game Boy’s four shades and the CRT’s three color stripes, sits at the two ends of the same aesthetic curve.

Thanks for reading! Follow me if you'd like~

Sixteen Milliseconds per Frame

Sat Jan 10 2026

8144 words · 42 minutes

Frontend Engineering Games Performance