Generative Systems: Cellular Automata, Markov Chains, and Procedural Generation

Generative systems share one creed: complexity need not be designed — it can be grown. Part one watches cellular automata conjure global order out of local rules that look only at their neighbors; part two presses language through a Markov chain into a count table; part three watches procedural generation lay a layer of controlled correlation over pure randomness. All three share one toolbox — grids, random numbers, sampling, and mapping — and one engineering bottom line: reproducibility.

Cellular Automata: Local Rules, Global Order

Global behavior. Gliders drifting, fire lines advancing, epidemic wavefronts. None of it is written anywhere in the code; all of it is an emergent product of local rules. This chapter follows the escalating line of deterministic cellular automata, probabilistic lattice models, and compartment equations, looking at what questions each class of model answers best, and the key traps of implementing them yourself.

Hands-on versions: Cellular Automata, Fire Spread Simulation, the SIR Epidemic Model, and Ecosystem Sim.

The four-piece kit

Any cellular automaton (CA) is defined by four things.

Grid. One, two, or more dimensions.

State set. Each cell’s possible values, say alive/dead two-state.

Neighborhood. Who each cell can see. The von Neumann neighborhood takes the four cells up, down, left, and right; the Moore neighborhood adds the four diagonals for eight in total.

Update rule. Based on its own state and its neighbors’, decide its state at the next tick.

All rules are local. A cell knows nothing of cells a thousand miles away; information travels one cell at a time.

One more convention, easy to overlook and utterly crucial: synchronous update.

All cells compute their next state from the same snapshot at once, rather than sweeping left to right and changing each one as you compute it.

The consequences of this convention wait for the implementation section; for now, remember: synchronicity is part of CA semantics — miss it and you are simulating a different system.

The Game of Life

Conway’s 1970 Game of Life compresses the four-piece kit to the extreme.

Two-dimensional square grid, two states (alive/dead), the Moore eight-neighborhood, three rules. A live cell with exactly 2 or 3 live neighbors survives, a dead cell with exactly 3 live neighbors revives, everything else dies.

That is the famous B3/S23 notation.

No random numbers in the rules, no hidden state; every initial pattern’s fate is fully determined mathematically.

Yet determinism and predictability part ways completely here.

Given the rules, can you tell whether a 50×50 pattern is dead or alive ten thousand steps later without watching it evolve? There is no shortcut. The only way is to run it.

Wolfram classified cellular automata’s long-run behavior into four classes: settling into uniformity (total extinction or total stability), falling into cycles, complete chaos, and the most fascinating fourth class — local structures stably persisting and interacting on a chaotic background.

Life belongs to the fourth class. The Glider is a five-cell pattern translating itself forward periodically, the Pulsar is a period-3 oscillator, and colliding gliders can annihilate, bounce, or produce new structures. Turing completeness was proven precisely amid these “structural chemical reactions”.

The site’s Cellular Automata game builds in both lines.

The LIFE tab is a 50×50 Game of Life with preset patterns like gliders and pulsars, plus random fill, hand drawing, and speed-adjustable playback.

The CA tab is a one-dimensional elementary cellular automaton. Each cell looks only at its left neighbor, its right neighbor, and itself; rules are encoded by numbers 0 to 255 (8 neighborhood combinations, each mapping to an output of 0 or 1).

A few of those numbers deserve a dedicated look.

Rule 30. Starting from a single black cell, it grows disordered chaotic lace. An old way of generating pseudo-random numbers; Mathematica actually once used it as its default random source.

Rule 90. The XOR rule; its evolution pattern is exactly Pascal’s triangle mod 2, the Sierpinski triangle growing straight out of the rule.

Rule 110. Structurally rich enough that Cook proved it Turing-complete in 2004. One rule among the one-dimensional elementary automata, and it can compute everything computable.

Wolfram gave this kind of system a name: computational irreducibility.

The fastest way to predict an irreducible system is to let it run. There is no formula shorter than the simulation itself.

This is not a compute-power issue; it is the ontology of complexity.

Probabilistic lattices

In the Game of Life every rule is deterministic, but the real world’s cells mostly carry dice. Whether this tree catches fire, whether this person gets infected — both are probability events.

Swap the deterministic verdicts in the update rule for Bernoulli trials and you get a probabilistic lattice model. Randomized Methods: From Throwing Darts at Pi to Markov Chains spoke of “using random numbers to carry random processes”; here it lands on every single cell.

The Fire Spread Simulation is the purest specimen.

The grid holds four states: empty ground, tree, burning, scorched earth. Each step, burning cells turn to scorched earth, and every tree checks its eight-neighborhood. So long as one neighbor is burning, it ignites with probability $p$. Scorched earth slowly grows back into tree with a small probability.

The spread probability $p$ is adjustable with a slider (about 0.05 to 0.7); click any tree to light the fire, and the interface live-displays the count of burning cells.

The depth of this toy lies in phase transitions.

Take the trees as randomly scattered on the grid with density $\rho$ and the fire as a fluid spreading along connected paths, and the problem instantly connects to percolation theory in statistical physics.

Percolation comes in two flavors. Bond percolation (each edge opens with probability) and site percolation (each cell exists with probability). On a two-dimensional square lattice, the four-neighborhood bond percolation threshold is exactly 0.5, the site percolation threshold is about 0.593, and this post’s scenario, eight-neighborhood site percolation, has a threshold of about 0.407. More neighbors, easier connectivity, lower threshold.

Below the threshold, the forest contains no tree chain crossing the whole region, and a fire, left to burn, necessarily extinguishes itself in the fragments. Above it, an infinite connected cluster from a point to the boundary exists, and fire sooner or later burns through the whole field.

The site’s simulation starts at a tree density of about 0.7, far past the threshold, so any single ignition may become a conflagration.

Drag the spread-probability slider very low and you have attached random blocking to every “physically connected” path. Effective connectivity drops, and the fire line shatters into isolated fire spots.

Macroscopic outcomes respond to microscopic parameters not linearly but as threshold-like jumps. This is a phenomenon probabilistic lattices can demonstrate and deterministic CAs cannot.

One honest finite-size disclaimer is owed too.

Thresholds like 0.407 are strict limits of the infinite lattice; the “critical density” measured in a 60×60 simulation shifts with boundaries and fluctuations. And the burned area of a single run is itself extremely widely distributed (same parameters: some seeds burn 5%, some 60%).

When reporting conclusions from lattice simulations, always attach “how many runs, and what the distribution looks like”. A single number will lie.

Where probabilistic lattices and CAs part ways

Note where the dividing line sits. The structure is completely identical (grid, neighborhood, synchronous update); the only difference is whether the update rule contains a random term.

In the deterministic Game of Life, the same initial pattern always evolves the same trajectory. In a probabilistic lattice, every run is a new trajectory.

So “prediction” changes meaning from “compute the outcome” to “compute the outcome’s distribution”. To answer “what fraction of the forest can this fire burn”, the right move is to run a hundred ignitions and draw a histogram, not to hunt for a formula.

Monte Carlo thinking plugs in right here. One simulation is a sample; many simulations are statistics.

Compartment models

One step further from the probabilistic lattice and you reach compartment models. No longer track every cell; count only the totals per state, and average contacts between individuals into a single proportional term.

This is epidemiology’s classic SIR equations:

$S$, $I$, $R$ are the counts of susceptible, infected, and removed (recovered or dead); $\beta$ is the transmission rate, $\gamma$ the removal rate.

The model’s entire early-warning power condenses into one dimensionless number: $R_0 = \beta/\gamma$ — how many people one infectee infects on average in a fully susceptible population.

$R_0 > 1$ means exponential growth, $R_0 < 1$ means the epidemic dies out on its own; there is no third possibility.

The threshold itself can be derived in one line. An infectee’s average infectious period before recovery is $1/\gamma$; in that window they contact $\beta$ susceptibles per unit time, so each infectee creates $\beta/\gamma$ new infections on average. That is $R_0$.

It immediately yields a quantitative yardstick for intervention. Vaccine (or isolation) coverage $v$ presses the effective susceptible fraction to $1 - v$; requiring the effective reproduction number $R_0(1-v) < 1$ gives:

That is where the “herd immunity threshold” comes from. At $R_0 = 3$, more than two-thirds of the population must be covered.

An ODE model cannot give the shape of the wavefront, but this one inequality was (and remains) a core quantity in public-health decisions. A crude model’s analytic criterion often has more action value than a fine simulation’s curves.

The site’s SIR Epidemic Model simulation takes a middle route: compartment state semantics + lattice spatial structure.

On a 60×60 grid, each cell is one of S, I, R. Each step, infectees recover with probability $\gamma$ and, with probability $\beta$, attempt one by one to infect their susceptible neighbors in the four orthogonal directions. Click any cell to release the initial infection; the interface gives both the three-state counts and a live curve in the upper right.

It deviates from the pure ODE version in two thought-provoking ways, and those deviations are exactly the value of spatial structure.

Early stochastic extinction. In the ODE, so long as $R_0>1$, whether the initial infection is 1 person or 10,000, the curve necessarily takes off. In the lattice version, the first few infectees may all happen to recover while every surrounding neighbor happens to escape. The epidemic dies naturally in infancy.

That probability of “escaping by a hair” does not exist in the ODE at all and is clearly visible in the lattice simulation. Why doesn’t every sneeze spark an epidemic? This is the answer.

A wavefront, not homogeneous mixing. The ODE assumes anyone contacts anyone with equal probability (complete mixing); in the lattice version, infection can only advance cell by cell. You will see a clear ring of wavefront pushing outward at a roughly constant speed, with an exhausted immune zone behind it.

Spatial structure corrects homogeneous mixing’s overestimate. Neighbors get repeatedly exposed to the same infectee, and effective transmission is diluted by local clustering.

Both deviations point at the same technical term: mean-field approximation.

The ODE’s $\beta SI/N$ term assumes each susceptible’s infection probability is the global infection density — stirring the grid smooth and taking the average. That is exactly why it is called a compartment model: only the total flows between the three compartments are watched.

What mean-field erases is precisely spatial structure’s most valuable output (wavefronts, clusters, boundary effects), so the modeling choice is essentially pricing “analytic tractability” against “spatial fidelity”.

The ecosystem tank equations

Discard even space entirely and you get the third class’s extreme form: discrete integration of pure compartment equations.

The Ecosystem Sim tool is just such a system.

Three stocks — plants, herbivores, carnivores — and three environment sliders — light, water, food. Each turn pressed, the program integrates one step of a system of logistic-growth-plus-predation equations.

Plant growth is suppressed by their own size (the logistic term $1 - P/K$, with carrying capacity $K$ set by light and water) and grazed by herbivores through a saturating function. Herbivore growth comes from eating, their death rate is modulated by food and water, and they lose numbers to predation. Carnivores likewise.

Numerically it is a modern adaptation of the Lotka–Volterra predator–prey equations. That pair’s most famous property is the limit cycle: predator and prey numbers rise and fall against each other, never at rest.

The lattice SIR’s curve is a single wave rushing to its end, while the population curves here are closed loops that can keep circling forever.

Why the limit cycle turns is worth taking apart for a look at the dynamics.

More prey → predators well fed, numbers rising with a lag → predation pressure crushes the prey → prey crash, predators go hungry, numbers falling with a lag → predation pressure eases, prey rebound.

The two keywords are lag and saturation.

The predator’s response always trails the prey by half a beat (reproduction takes time, which in the equations shows up as the predation term acting only on the predator’s growth rate), and predation saturates as prey density rises (expressed in saturating functions like $\frac{H}{H+k}$).

Remove that half-beat lag and the cycle collapses into a straight line toward equilibrium. The oscillation is a structural property, not a parameter coincidence.

In the ecosystem tank, maxing out light and food often shows plant mass shooting to the ceiling and then dragging the whole food chain down with it. Herbivore boom → plant collapse → carnivores starving in a chain → the tank resets. That is an overshoot manufactured jointly by the logistic ceiling and the predation lag.

The real-world greenhouse’s “boom-bust” management problem is exactly this at the equation level.

The three model classes

Put the three lines together and the trade-offs become very clear.

The three are not a progression but a division of labor.

Deterministic CAs answer “where does structure come from”. Probabilistic lattices answer “where are the probability thresholds”. Compartment equations answer “what does the long run look like”.

The popular modeling order follows exactly this line. Use the ODE first to catch the main contradiction (compute $R_0$ and you know whether to intervene), then go to the probabilistic lattice for the deviations that space and randomness bring (wavefronts, clusters, early extinction), and only reach for the deterministic CA when the emergence mechanism is itself the object of study.

The more dimensions erased, the cheaper and more analytically tractable the model. The more dimensions kept, the truer the model and the more it depends on simulation.

Implementation notes

Writing any lattice simulation yourself, three pits are nearly unavoidable.

Synchronous update requires double buffering.

All cells must compute their new state from the same-moment old state, so first allocate a next grid, write the whole round’s results into it, and swap the pointer at the end. Change cells in place as you sweep and the cells swept early “pollute” the neighbor view of the cells swept later.

How deep this pit goes deserves a concrete example. Fire spreads left and right with equal probability, but a single buffer sweeping rows left to right ignites the left-side cells first, and that round’s newly ignited fire immediately participates in the right-side cells’ verdict (they read already-updated values), so the fire visibly runs only rightward. A directional bias with no physical reason whatsoever, purely an artifact of traversal order.

The correct implementation is a dozen-odd lines:

1
function step(grid, W, H) {
2
  const next = new Uint8Array(grid.length);        // the new world
3
  for (let y = 0; y < H; y++) {
4
    for (let x = 0; x < W; x++) {
5
      let n = 0;                                   // neighbor count: read only the old world
6
      for (const [dx, dy] of NEIGHBORS) {
7
        const nx = (x + dx + W) % W, ny = (y + dy + H) % H;
8
        if (grid[ny * W + nx]) n++;
9
      }
10
      next[y * W + x] = rule(grid[y * W + x], n);  // write only the new world
11
    }
12
  }
13
  return next;                                     // swap wholesale
14
}

Every simulation mentioned in this post follows this pattern without exception: read old, write new, swap wholesale.

Boundary semantics need thinking through.

The top-left corner cell has two neighbors fewer. What rule does it follow?

Two choices are common. Wrapping (join the grid’s edges up, down, left, right into a torus — the modulo in the code above does exactly this) or walls (out of bounds counts as empty ground/dead cell).

Neither is wrong, but they give different phase-transition thresholds. The 0.407 percolation threshold is the infinite plane’s value; a small grid with wall boundaries measures a higher critical density.

When running experiments, boundary semantics belong in the conclusion’s footnote.

Decouple rendering from simulation.

Simulation stepping and screen refresh rates don’t match; the standard move is to count frames in the animation loop and advance the simulation only every several frames. With many cells or expensive rules, this saves the frame rate.

Use fixed-length typed arrays (like Uint8Array) instead of nested arrays, addressing directly with the one-dimensional index $i = y \times W + x$. For simulations of hundreds of thousands of cells, that is an order of magnitude.

Scale further up (city-scale, hundreds of millions of cells) and the two weapons are sparse storage and parallelism. Maintain a hash set of only the active cells and their neighbors (in the Game of Life the vast majority of cells are dead zones long-term), or move the whole grid onto the GPU and let one thread per cell compute independently.

“Each cell depends only on its local neighborhood” is literally the CA’s definition, so it natively suits massive parallelism. That is also the engineering reason it remains active on the front lines of fluid, traffic, and materials simulation.

These three are all engineering details, but lattice simulation’s “scientific credibility” hides inside them. Get double buffering wrong and what you observe is not the model’s properties but the bug’s properties.

Swap the deterministic verdict of a next state for a probabilistic draw, and language can grow out of the grid. The next chapter swaps cells for words.

Markov Chains: Next-Word Prediction in Miniature

Text generated by a Markov chain has a distinctive air. Every word joins seamlessly, yet three sentences in you have no idea what you just read. The nonsense literature of “after hearing your words, it feels like hearing your words” is, statistically, an exact portrait of this model class.

But this toy, born in the 1950s, deserves today’s engineers’ serious attention. In a few dozen lines of code it demonstrates every gear of the great machine called “next-word prediction”, and large models have not replaced those gears — they only changed the drive from counting tallies to neural networks.

Hands-on versions: Markov Chain Text, Writer Toolkit.

One Chain, One Matrix

The Markov chain’s core assumption is a single one: the probability of the next state depends only on the current state.

Moved onto text, the state is a word (or a character), the “current state” is the last one or two words, and the transition probabilities are counted out of a corpus.

Building a text generator takes only five steps.

Tokenize. Cut the corpus into tokens. Cutting by character gives a small vocabulary and connects anything to anything, but the sentences come out more fragmented. Cutting by word raises fluency at once, but demands a far larger corpus. Chinese has no natural space tokenization, so a character-level model is the zero-cost starting choice.

Count. Sweep the corpus once, counting every adjacent word pair $(w_i, w_{i+1})$ into a sparse table. A second-order model counts triples $(w_{i-1}, w_i, w_{i+1})$ instead.

Normalize. Divide each state’s counts by that state’s total number of appearances to get transition probabilities. Or skip normalization entirely and sample directly weighted by counts — equivalent, and one floating-point step cheaper.

Generate. Start from a start state and loop “look up the table → sample the next word by probability” until you draw a terminator or hit the length cap.

Patch the gaps. On hitting a “no successor” dead end (this word never once appeared mid-sentence in the corpus), fall back to a lower-order model or a random restart.

The JavaScript implementation is short enough to paste whole:

1
function train(tokens, order = 1) {
2
  const table = new Map();
3
  for (let i = 0; i < tokens.length - order; i++) {
4
    const key = tokens.slice(i, i + order).join(" ");
5
    const next = tokens[i + order];
6
    if (!table.has(key)) table.set(key, {});
7
    table.get(key)[next] = (table.get(key)[next] || 0) + 1;
8
  }
9
  return table;
10
}
11

12
function generate(table, start, order = 1, maxLen = 30) {
13
  const words = start.split(" ");
14
  for (let i = 0; i < maxLen; i++) {
15
    const key = words.slice(-order).join(" ");
16
    const dist = table.get(key);
17
    if (!dist) break;                 // dead end: fall back or terminate
18
    words.push(weightedPick(dist));   // sample weighted by counts
19
  }
20
  return words.join("");
21
}

You can see what the matrix’s skeleton looks like directly in Markov Chain Text.

It offers an editable three-state transition matrix (each row’s probabilities auto-normalized); one click generates a 30-step state sequence and tallies how often each state appears along that trajectory.

Read the matrix as a text generator’s corpus. The more concentrated a row’s probabilities, the more fixed “what follows this word”. The more uniform the rows, the more the generation drifts.

Change one cell and click generate once more, and you can watch with your own eyes how a tiny change in transition probability gets amplified into a completely different trajectory over a long sequence.

Four Knobs

All the secrets of generation quality sit on four knobs, each one a trade-off with its price plainly marked.

Order

A first-order chain looks only one word back, so local collocations stay loose. Second order looks two words back and fluency jumps. Third order looks three words back.

Fluency keeps climbing, but the state count explodes exponentially in vocabulary size. On a fifty-thousand-word corpus, the second-order theoretical ceiling is $50000^2 = 2.5$ billion states.

Deadlier still is sparsity. Combinations never seen in the corpus have probability zero, and the generator hits a wall the moment it steps onto a path it has not memorized.

So practical engineering forever haggles between “order for fluency” and “sparsity for coverage”. The same seesaw survives in neural language models under new names (context window, parameter count).

Smoothing

Zero probability is the counting model’s fatal spot; smoothing is the collective name for the countermeasures.

The simplest is add-one smoothing (Laplace smoothing). Add 1 to every count and renormalize — carving a small slice out of the total probability for combinations “never seen”, at the cost of pressing down the high-frequency ones.

Engineering more often uses interpolated smoothing. When the third-order model finds nothing, fall back to second order, then second falls back to first, mixing the three orders’ distributions with fixed weights. The backoff idea.

Whether the smoothing parameters are tuned well or badly directly decides whether the generated text is “occasionally off-style” or “outright gibberish”.

Temperature

Sampling temperature is the most direct legacy this toy leaves to large models.

Transform the pre-normalization logits (here, the logarithms of the counts):

$T = 1$ is the original distribution. Shrink $T$ and probability contracts toward the most frequent words; $T \to 0$ degenerates into always picking the largest (greedy). Grow $T$ and the distribution flattens, cold words gain their chance; $T \to \infty$ becomes uniform blind sampling.

Three lines of code, a creativity knob continuously tunable from “repeater” to “madman”.

The temperature parameter you see in any large-model API today is the same math for the same purpose — only the logits come from a neural network’s output layer instead of a count table.

Temperature’s effect is clearest with a set of concrete numbers.

Say a state has three candidate words with counts 5, 3, and 1; taking logarithms gives logits $[\ln 5, \ln 3, 0] \approx [1.61, 1.10, 0]$; divide by different temperatures and renormalize:

Low temperature magnifies the gaps, high temperature compresses them, and the ranking never changes. Temperature only reshapes confidence, never the order of preference.

That explains two phenomena from practice. Low-temperature generation is stable but gives itself away quickly (the same opening walks into the same sentences); high-temperature generation is fresh, but the collapse rate rises.

Tuning temperature is essentially sliding between “faithfulness to the corpus” and “the ability to step off it”.

Truncation

Temperature reshapes the distribution’s form but cannot touch the candidate set itself. Tens of thousands of vocabulary words all still take part in sampling, and the long tail, however cold, still clutches a sliver of probability.

Truncation supplies exactly this cut.

top-k keeps only the top k by score, zeroes the remaining probability mass, and renormalizes.

top-p (nucleus sampling) goes a step further. Sort the candidates by score descending, accumulate probability from the highest, and stop once the sum reaches p (say 0.9), sampling only inside this “core set”.

The difference lies in adaptivity. k is a fixed number, so when the distribution is sharp (after “New”, “York” almost monopolizes the mass) those k slots keep a crowd of near-zero escorts in the running; when the distribution is flat (free improvisation at a sentence start), k may not be enough.

top-p’s candidate count stretches and shrinks with the distribution’s shape — two or three candidates left at sharp spots, dozens released at flat ones — which is why it has become one of the defaults in today’s large-model APIs.

The division of labor between truncation and temperature deserves its own sentence. Truncation decides who is allowed on stage (it changes the ranking; the truncated are simply out), temperature decides how those on stage split the votes (it does not change the ranking, only the gaps).

Truncate first, adjust temperature second — the standard pipeline of large-model decoding.

And every section of that pipeline can be built as-is on an n-gram generator. Sort by counts and chop the low-frequency tail, that is top-k; raise the counts to the $\lambda$ power and renormalize, that is temperature, and the reciprocal of $\lambda$ is exactly $T$.

The statistical toy and the large model are fully isomorphic here; only the scale of the candidate set differs.

Walking Through It

Put the five construction steps onto a real, tiny corpus — small enough to hand-compute end to end.

Six sentences:

1
cat chase ball
2
cat eat fish
3
dog chase cat
4
dog eat bone
5
cat sleep sofa
6
ball roll around

After character/word splitting (space tokenization here), count the first-order bigram table:

Sample a path starting from “cat”. Look up the “cat” row and draw “chase” at equal probability. Look up the “chase” row and draw “ball”. Draw “roll”. Draw “around”. Hit a dead end, trigger the patch rule (fall back to a random restart), then continue from an arbitrary word.

The generated “cat chase ball roll around” never actually appears whole in the corpus. It is a recombination of legal fragments — which is the mechanistic root of Markov generation being at once “fresh” and “nonsense”.

Note also how universal dead ends are. Four of the eleven words have no successor (fish, bone, sofa, around), so on a small corpus the patch strategy fires more often than the sampling itself.

In real engineering, “put the period into the table as an ordinary token” or “split the corpus by sentence, each sentence its own chain” exists precisely for this.

An Information-Theoretic Diagnosis of Nonsense Literature

First, a checkup for this toy, with information theory as the instrument.

Shannon estimated English’s entropy in 1948 using $n$-th order approximations, and in 1951 designed a clever human experiment. Subjects guessed an English text letter by letter, a correct guess advancing to the next letter, and the distribution of guesses needed maps directly onto the text’s “unpredictability” at each position.

Arranged by model order, the results form a beautiful descending staircase. Zeroth order (letter frequencies known) comes to about 4.14 bits per letter, first order (looking one letter back) about 3.56, second order about 2.89. Humans, drawing on the full context and common sense, press the number down to around 1 bit.

The staircase is steep at low orders and gentle at high ones, meaning statistical regularity eats the bulk of the redundancy. But that last stretch — “semantics and common sense” — is exactly the territory n-gram models can never reach.

Shannon’s experiment left a byproduct that still holds. The text sequences he hand-”generated” with the same high-order approximations (asking a subject to complete each step by feel) could already fool readers’ expectation of “this is English”. The correspondence between generation quality and context utilization was quantitatively nailed down back in that era.

Language’s heavy informational redundancy is the reason statistical models can imitate it. Mimic the redundancy correctly and it sounds like speech.

Now “why does it sound like speech yet carry no content” can be answered precisely.

What the Markov chain preserves is local statistics. A second-order model memorizes every adjacent collocation in the corpus, so every word’s joining passes the ear.

What it throws away is long-range dependency. How the sentence opened, what claim the paragraph staked, who the subject of chapter three was — none of that fits into a “last two words” state.

The generated text is a trajectory doing a random walk through semantic space. Every step stays on the rails; the whole goes nowhere.

Topic drift, scrambled pronouns, “fluently answering the wrong question” — all different profiles of the same defect. Strong back-edges in the transition matrix also set the trajectory spinning periodically: when “so-and-so” appears often in the corpus, looping phrases like “so-and-so says, so-and-so laughs, so-and-so says” come out in strings — a random walk fallen into an attractor.

So the direction of improvement is naturally obvious: enlarge the state.

Swap “the last two words” for “a vector representation of the whole context”, and swap “table lookup” for “one nonlinear mapping of that vector”. The next-word-prediction frame does not move an inch, but the state’s capacity goes from vocabulary-squared to network parameter count.

The n-gram’s “previous word” is a discrete index: two different contexts are either fully equal or fully different, with no middle ground.

A neural language model’s context vector, by contrast, lives in a continuous space. “Weekend picnic in the park” and “holiday camping in the countryside” map to nearby positions, so having seen one, you can make a decent prediction about the other. Generalization comes from that continuity, something a lookup model cannot have in principle.

Attention’s contribution is to add learnable weights on “which parts of the context are worth reading”. Not a fixed window but dynamic selection by relevance, turning long-range dependency from “won’t fit in the state” into “retrieved on demand”.

There is one more easily overlooked junction: where the tokens come from.

Splitting in the Markov era (by character, by word) was decided by hand; neural language models use subword algorithms like BPE (Byte Pair Encoding). Starting from characters, iteratively merge the corpus’s most frequent adjacent pairs, keeping frequent words whole and splitting low-frequency ones into subword fragments.

It inherits the n-gram’s statistical soul (the merging criterion is co-occurrence frequency) while solving the dilemma of vocabulary size versus unknown words. Any new word decomposes into a platter of known subwords.

From Shannon’s letter-level experiments to BPE, the question “how coarse a grain to cut text into” has been answered over and over for seventy years. The answer sways with compute and data scale; the question itself never changes.

The stack after that leap is wrapped layer upon layer in today’s large-model application frameworks. The earlier post Eino Study Notes recorded the Go ecosystem’s way of orchestrating it; behind ChatModel’s one-question-one-answer, what the model layer still delivers is “the probability distribution of the next token”.

Temperature, top-k, top-p — these sampling parameters are inherited verbatim from the statistical language model era.

What large models retired was the table lookup, not the framework.

Why It Wasn’t Retired

Since large models crush it across the board, why does this toy still live in production systems? Three reasons, each harder than the last.

Zero Compute, Runs Locally

A chain’s entire overhead is one count table plus one weighted sample.

The table builds in tens of milliseconds in a browser and runs just as well on a microcontroller — no network requests, no GPU, no per-token billing.

Random NPC barks in games, name generators, corpus-morphed test data, local text transforms for privacy-sensitive scenarios. These scenarios’ demand for “intelligence” sits far below their demand for “zero dependencies”; the n-gram is their optimal solution, not a compromise.

Two high-frequency use cases deserve naming.

In fuzzing, train a chain on the target system’s input corpus; the “almost plausible” malformed payloads it generates reach deeper into a parser’s branches than pure random noise does.

In game content generation, short texts that must “stay in style but never repeat” — NPC names, tavern notices, random verse — a small chain is four orders of magnitude cheaper than calling a large model.

Explainable, Auditable

A neural network’s transition probabilities are buried in hundreds of millions of parameters; an n-gram’s transition probability is one cell in a table. Every number can be read, questioned, and modified directly by a human.

To audit a model’s behavior (“why does it always generate this word”), look up the table. To inject a prior (“this word must never be followed by that one”), change one cell.

Explainability is not merely a virtue; it is an admission ticket in some industries. Compliance review has to answer “where did this output come from” — a table lookup yields an evidence chain, while a neural network mostly yields “the gradient was large”.

Content safety is the same. Sensitive-word filtering on an n-gram is deleting or downweighting a few table entries, with fully predictable behavior. On a black-box model it is an ongoing adversarial slog.

Controllable, Injectable

This is exactly the dividing line between it and human creative tools.

The site’s Writer Toolkit takes the other road. Name generation, worldbuilding cards, character cards, inspiration dice, plot templates, word counts, writing-sprint timers. Every module is a hand-curated material library plus deterministic combination — no sampling, no random walk; you ask for something and it hands you exactly that.

Holding the two roads side by side is very instructive.

The Markov chain is “the machine discovering probabilities in the corpus by itself”; the Writer Toolkit is “a human writing the probabilities down in advance”. The former runs unchecked but often throws up happy accidents; the latter stays controllable but needs constant feeding with material.

Generation systems in practice usually want both ends. Priors control the structure (outlines, constraints, templates); randomness fills in the flesh (word choice, variants, surprises).

In this hybrid architecture the n-gram still carries every “cheap but random” segment. The constraints stay in human hands; the chain only walks every possibility inside them.

Practical Boundaries

Finally, a few details that will save you half a day once they are written down.

Chinese tokenization. Character-level models are zero-dependency but fragmented; word-level needs a segmenter; the compromise is starting from bigrams of characters.

The UTF-16 trap. JavaScript’s split("") cuts emoji and rare characters into orphaned surrogate pairs, so you must split with Array.from() or Intl.Segmenter, or the generator outputs mojibake.

Sentence boundaries as state. Mixing periods into the chain as ordinary tokens fuses the generated sentences into a single blob. The better approach is to split the corpus by sentence and make each sentence its own chain, turning termination into part of the structure.

Loop detection. When the transition matrix has strong back-edges (many high-frequency phrases in the corpus), generation gets stuck in “so-and-so so-and-so so-and-so” repetition. Record the last $n$ steps and re-sample or force a lane change on a hit, and you catch even the most embarrassing moments.

Performance. Once the corpus grows, a dense two-dimensional count table is a memory black hole (a 50,000-word vocabulary’s second-order model has a theoretical ceiling of 2.5 billion cells). A sparse Map with keys built on demand is the only viable answer; during table building, a single pass that reads the corpus as a cursor array instead of copying strings can differ in memory footprint by an order of magnitude.

And don’t evaluate by eyeball alone.

Language models come with a ready-made ruler: perplexity. Take a held-out test text, have the model predict word by word, take the negative logarithm of the true word’s probability at each prediction, average, and finally exponentiate.

Intuitively it is “the model’s average number of effective candidates at each fork”. Perplexity 20 means every position looks to the model like 20 words are competing.

It is precisely the mathematization of Shannon’s letter-guessing experiment. In 1951 it took human guess-counting; today one line of code yields a reproducible metric.

Whether generated text reads well is subjective; whether perplexity fell is objective. Watch the metric first and the experience second, or tuning turns into folklore.

Count tables can generate sentences that sound right, but not terrain that looks right: the role of randomness inverts here, from something to let loose to something to rein in. The final chapter asks exactly that — how much correlation to add before noise grows into something worth looking at.

Procedural Generation: Noise, Fractals, and Terrain

The core contradiction procedural generation has to resolve fits in one sentence: randomness brings richness, but it also brings shattering.

A purely random pixel image is a sandstorm, while the terrain a game needs should have rolling mountain ranges and coherent coastlines, and its textures natural mottling rather than uniform speckle.

The field’s foundations run in three steps: build smooth noise first, then stack noise at several scales proportionally into a fractal, and finally map the numerical field onto color or geometry — every step of that chain can be twiddled by hand in a browser.

Hands-on versions: Texture Generator, Fractal, 3D Terrain, Voronoi, Lissajous, Plasma, Metaballs.

Starting from white noise

Start with the cheapest counterexample.

The Texture Generator has two modes. “Noise” mode rolls one uniform random number independently for each pixel of a 240×240 grid, and that is white noise: zero correlation between any two pixels, a power spectrum flat across every frequency band like white light (whence the name).

As a texture it reads as “TV static”, as terrain it is a disaster, and as raw material it is indispensable. Test patterns, jittered sampling and grain filters all rely on it.

The other “waves” mode slides to the opposite extreme: $\sin(x\pi/s) \cdot \sin(y\pi/s)$, the product of two orthogonal sines. One frequency, one period across the whole field, perfectly tileable. This is a regular texture: good-looking but fake, machine-drawn at a glance.

Natural textures live between the two extremes. Neighboring pixels are correlated (not static), but the frequency content is distributed continuously over a range (not a single sine).

The two rulers for this “in-between state” are correlation length (how far apart two pixels have to be before they stop recognizing each other) and spectral shape (how energy decays with frequency).

All of procedural generation’s engineering is producing a field of “specified correlation structure” with as little compute as possible. The next two sections cover the two mainstream routes: interpolate for correlation, stack for spectrum.

There is one tempting but wrong shortcut worth ruling out first: doesn’t blurring white noise give it correlation?

Blurring is indeed a low-pass filter and does kill high-frequency speckle, but the price is cutting the spectrum into two steps — white below the cutoff, nothing above. Real natural textures have power-law spectra, with energy at every frequency. Blurred white noise looks like a dispirited wad of cotton wool: the fine grain is gone, and there is no long-range structure either.

The correct route runs the other way: start from the low frequencies and add detail layer by layer. That is exactly what interpolation and fractal stacking do in the next section.

The other road to frequency synthesis comes from signal processing: lay out random phases and amplitude spectra for each frequency directly in the frequency domain, run an inverse Fourier transform, and get a field of the specified spectrum in one step. It is mathematically the most exact, but it needs an FFT and is hard to evaluate locally, so in real-time rendering it almost always yields to a per-pixel noise function.

Interpolation is the soul

The most direct way to turn white noise into structured noise is random first, smoothing after.

Store random numbers on a sparse grid of points (the lattice), and the value at any point is interpolated from the random numbers at the four neighboring lattice points.

The source of the site’s 3D Terrain takes exactly this route. Fill an 80×80 random lattice from the seed first, then bilinearly interpolate at sample time: along $x$, linearly interpolate the top and bottom edges into two intermediate values, then along $y$ interpolate those two into the final value.

This construction is called value noise. The values are continuous, but only $C^0$ continuous rather than $C^1$. At lattice-line crossings the derivative jumps, leaving grid-aligned artifacts that an expert identifies at a glance as “that’s value noise”.

Perlin noise — developed by Ken Perlin for the 1982 film Tron, published with a paper in 1985, and later awarded an Academy Award for Technical Achievement, a rare honour in graphics — does not replace the interpolation formula. It replaces what is stored at the lattice points.

Not random values, but random gradient vectors.

The noise value at any point = the dot product of each neighboring lattice point’s gradient vector with the offset vector from that lattice point to the sample point, then a weighted blend.

The dot product is always zero at the lattice point itself, so the noise field naturally has a zero contour through every lattice point. And the directional information makes the transition across lattice lines continuous in gradient. Combine that with a quintic smoothing curve $6t^5 - 15t^4 + 10t^3$ replacing linear interpolation (which keeps the second derivative continuous too), and the artifacts essentially disappear.

The difference between the two generations of noise in one sentence: value noise interpolates height, Perlin noise interpolates slope. At the same lattice density the latter is more isotropic and closer to “directionless nature”.

Engineering has two more questions that must be answered.

First, the lattice is only 80×80, so what happens when sampling coordinates double out of bounds?

A real implementation doesn’t reserve memory for an “infinite lattice”. It replaces lattice values with a hash of the integer coordinates: $\mathrm{hash}(x, y) \to [0,1)$. Arbitrarily large coordinates produce lattice values on the spot, and the field’s domain becomes the whole plane. The quality of the hash function directly determines whether the noise has a visible period.

Second, what if the texture has to tile?

The laziest approach is taking lattice coordinates modulo the period, so interpolation joins head to tail automatically at the boundary, at the cost of imposing an artificial period on the field. Game textures almost all use this trick, since GPU sampling is cyclic to begin with.

One textual matter to settle along the way: the terrain tool’s blurb says “Perlin noise generation”, but the implementation is value noise plus the fractal stacking that follows. That isn’t a bug; it is the best possible footnote — once enough octaves are stacked, the difference between the two noises in the final image is drowned by detail, and engineering often needn’t split hairs.

Simplex noise, which Perlin later introduced, swaps the square grid for a simplex grid with lower compute cost and better high-dimensional behavior, but that is an optimization, not a paradigm change.

Fractal Brownian motion

Noise of a single frequency, value or Perlin alike, still doesn’t look natural enough.

Real terrain has both large-scale mountain undulation and small-scale rock roughness.

The remedy is sampling the same noise at several scales and summing with weights: fractal Brownian motion (fBm).

The terrain tool’s core loop, simplified, is eight lines:

1
let amp = 1, freq = 1, sum = 0, norm = 0;
2
for (let o = 0; o < 5; o++) {          // five octaves
3
  sum += bilinear(lattice, x * freq, y * freq) * amp;
4
  norm += amp;
5
  amp  *= 0.5;                          // halve amplitude (persistence)
6
  freq *= 2;                            // double frequency (lacunarity)
7
}
8
height = sum / norm;                    // normalize

Each “octave” doubles the sample coordinate (halves the detail size) and halves the weight (shrinks the contribution). After five octaves the field holds undulations at scales 1, 1/2, 1/4, 1/8 and 1/16 at a weight ratio of $1 : \tfrac12 : \tfrac14 : \tfrac18 : \tfrac1{16}$.

The common names for these parameters come from signal processing.

persistence controls how much high-frequency content survives. The “roughness” slider in the source UI (0.2 to 1) multiplies the final stacked result directly, so pushing it up amounts to adjusting overall contrast.

lacunarity is conventionally fixed at 2, because powers of two make different octaves land exactly offset on the same lattice, reusing one table.

At persistence 0.5 the energy decays as $1/f^2$, the spectral signature of Brownian motion. That is where the “Brownian” in the name comes from. Push it near 1 and the high frequencies catch up with the low ones and the image turns bristly. Squash it below 0.2 and only the big skeleton is left, mountains flattening into gentle hills.

This stage also carries one easily overlooked engineering decision: determinism.

The terrain tool doesn’t use Math.random(). It implements a 32-bit xorshift generator (x^=x<<13; x^=x>>>17; x^=x<<5) to fill the whole lattice from the seed. So setting the “random seed” slider (1 to 999) back to the same number reproduces the terrain pixel for pixel.

The first engineering iron law of procedural generation systems is reproducibility. Content must be uniquely determined by the (seed, parameters) pair, otherwise tests can’t assert, people can’t share, and changes can’t roll back.

The choice of random number generator is secondary. xorshift needs a few dozen clock cycles and 4 bytes of state, and in a hot loop doing “five interpolations per pixel”, speed is a feature.

From height field to terrain

With a height field $h(x,y) \in [0,1]$ in hand, what’s left is turning it into a picture that “looks like terrain”.

The terrain tool chose isometric projection. A lattice point $(x, y)$ maps to screen coordinates:

where $t$ is the on-screen size of each cell (the “grid size” slider, 8 to 26).

The first line rotates the grid 45° and flattens it into diamonds. The cleverness of the second is the minus sign: the larger the height $h$, the smaller the screen $y$, so mountains are drawn in the negative direction. The 1.1× is a deliberately exaggerated vertical scale (otherwise the grain-of-rice height differences across an 80×80 grid would be invisible).

Drawing proceeds in ascending $x + y$, the painter’s algorithm’s old rule. Draw the far (upper left) before the near (lower right), and later tiles naturally cover earlier ones, so isometric terrain occludes correctly without a depth buffer.

Shading is a linear map of four coefficients. In the dark theme the color is $\mathrm{rgb}(20 + 0.25c,\ 70 + 0.5c,\ 110 + 0.45c)$, where $c$ is the height byte from 0 to 255. The three channels have different slopes — deeper blue down low, brighter cyan up high — and the green channel’s slope is the largest, so the “sense of height” is carried mainly by green brightness.

This 2D pseudo-3D is only two steps away from real 3D rendering, and understanding those two steps means understanding the whole spectrum of terrain rendering.

Step one replaces “one color block per cell” with “one face per cell”. Lift each set of four neighboring heights into four spatial points and connect them into a mesh, and it can enter WebGL and take lighting.

Step two maps height directly onto landform semantics. A sea-level threshold (water color below it), slope shading (bare rock where steep), vegetation bands by altitude (snow line, tree line). Real terrain generators almost all do their work inside this mapping function, while the height field itself is the cheapest part.

On top of the mapping you can stack physical simulation, the advanced class of modern terrain generation.

The height field is a marionette, and two classic simulation lines strap muscles onto it.

Droplet erosion sends virtual water drops downhill from high ground along the gradient, scouring soil away along the way and depositing it in the lows. After a few thousand drops, gullies are carved into the ridges and alluvial fans build the gentle slopes of valleys; that “carved by water” realizm is something no pure noise can dial in.

Tectonic uplift raises and sinks whole regions by a large-scale low-frequency field, manufacturing the grain of mountain ranges.

The simulation’s output feeds back into the height field, so the pipeline becomes three stages: noise as the base, simulation as the shaper, mapping as the paint. The site’s tool stops at the first stage, but open any world editor with terrain and you will recognize that three-stage skeleton.

Another small conclusion worth remembering is that shading and projection are decoupled. The height field is data, isometric projection is view, and switching projection (top-down, oblique 45°) needs no recompute of the field. The engineering common sense of separating data from view is worth just as much in graphics.

Two ways to draw a fractal

The Fractal tool packs seven fractals into one tabbed page, corresponding neatly to the two great schools of fractal generation.

The first school is escape time, and its protagonist is the Mandelbrot set.

For each pixel on the plane take a complex number $c$, iterate $z \leftarrow z^2 + c$ repeatedly from $z = 0$, and count how many steps it takes for the squared modulus to exceed 4 (escape radius 2):

1
while (iter < maxIter) {
2
  const x2 = zx * zx, y2 = zy * zy;
3
  if (x2 + y2 > 4) break;        // escaped
4
  zy = 2 * zx * zy + ci;
5
  zx = x2 - y2 + cr;
6
  iter++;
7
}

Points that don’t escape are painted dark (inside the set), and escaped ones are colored by step count. In the source the hue is $200 + t \times 480$ degrees, where $t$ is the iteration count over the cap of 150 — that is where the blue-violet-gold gradient bands in the pictures come from.

The initial window is centerd on $(-0.5, 0)$ with width 3.5, and can be dragged, zoomed with the wheel, and pushed to a millionfold zoom with new detail still turning up. “Structure at any magnification” is exactly the defining feature of a fractal.

The iteration cap is this school’s first performance knob. The cap sets the “boundary resolution”: the deeper you zoom, the more iterations boundary points need before they can be classified, and with too low a cap the boundary smears into color bands. And every zoom reruns the whole loop over every pixel, so the quadratic per-pixel cost makes progressive rendering (coarse grid first, refine after) standard kit.

The second invisible ceiling is floating-point precision. A 64-bit double has about fifteen or sixteen significant digits, and once the window shrinks to the $10^{-13}$ scale the coordinate difference between neighboring pixels is smaller than one float step and the image quantises into mosaic. Going deeper means extended precision or fixed point — the shared boundary of every fractal zoomer.

The Julia set next door uses the same iteration, only with the roles swapped. $c$ is fixed (two sliders adjustable within $\pm 1$, or five presets: Classic, Dragon, Spiral, Rabbit, Star), and $z$ starts from the pixel coordinate.

The relationship between the Mandelbrot and Julia sets is one of the most beautiful results in graphics: the Mandelbrot set is the parameter map of all Julia sets. Pick $c$ inside the Mandelbrot set and the corresponding Julia set is connected; pick it outside and the Julia set shatters into dust. Every shape change you see while dragging $c$ on the Julia page corresponds to the point $c$ moving in or out of the Mandelbrot set.

The second school is recursive substitution.

From the Sierpinski triangle (three half-size copies of itself) and carpet (cut the center out of a 3×3 grid and recurse into the remaining eight) to the fractal tree (each branch forks twice at 25° with a length ratio of 0.72, nine levels deep, and the randomise button just draws randomly inside that parameter space).

They share one formal language with L-systems. A rewrite rule substitutes into a string over and over, and then a “turtle” interprets the characters as drawing commands: F moves forward, +/- turns, [ and ] push and pop the stack to branch.

The Koch snowflake in the tool is one rule $F \to F+F--F+F$ plus ±60° turns rewritten four generations. The dragon curve $X \to X+YF+,\ Y \to -FX-Y$ uses 90° turns. The plant preset’s rules are busier, but the skeleton is the same.

These shapes’ “fractal dimension” can be computed exactly with the box-counting formula $D = \log N / \log(1/r)$. The Koch curve replaces each segment with 4 copies at 1/3 the size, so $D = \log 4/\log 3 \approx 1.26$. The Sierpinski triangle gives $\log 3/\log 2 \approx 1.59$. The carpet gives $\log 8/\log 3 \approx 1.89$.

The dimension is no longer an integer; the curve is “longer than one-dimensional, with less area than two-dimensional”, which is the mathematical version of the coastline paradox.

Drawing with distance fields

The last group of widgets shares one kernel: evaluate a scalar field or distance field for every pixel, with color as nothing more than a visualization of the result.

Voronoi has the simplest field. A crowd of randomly wandering seed points, each pixel painted the color of the seed nearest to it.

Brute-force nearest neighbor: every frame, each of the 120×120 pixels does $N$ distance comparisons (the point-count slider runs 3 to 40), and scaled up it becomes that image of ceaselessly wriggling, splitting cells.

Its mathematical definition (each seed’s cell is the set of points closer to that seed than to any other) is equivalent to an intersection of half-planes. Yet in engineering practice brute force is nearly the default: with $N$ in the tens, sweeping pixels is faster and less bug-prone than building a graph structure.

Voronoi’s application list is startlingly long. Provinces and spheres of influence on maps, honeycomb textures and cracked-mud simulation (Worley noise simply uses Voronoi distance as color), even the decision boundary of nearest-neighbour classifiers in machine learning — which is exactly a Voronoi diagram.

It also has a famous post-process called Lloyd relaxation. Move each seed to its own cell’s centroid, and after a few rounds the distribution tends toward uniform. In procedural generation, “scatter randomly, then relax to evenness” is the standard two-step for sampling that is both natural and even; star fields, resource nodes and vegetation spreads are all scattered this way.

Metaballs swaps the distance field for a potential field.

Ten orbs with fixed hues swim through the field, and each pixel sums over all ten:

Only pixels above the threshold count as “submerged in the fluid”.

The beauty is the inverse-square falloff. Neither orb alone clears the threshold, but when they come close the sum in between crosses it first, pulling a “bridge” between them. Orbs merge as they approach and stretch into threads that snap as they part — which was the entire motivation when Jim Blinn invented metaballs for molecular visualization in 1982.

The source has one piece of fine work as well. A pixel’s hue is the squared-influence-weighted sum over the ten orbs, so it leans toward whichever orb is nearest, and the merge zone therefore shows natural color transitions. The edge factor $(f - 1) \times 2.5$ at the boundary makes saturation and opacity fade across the contour, avoiding hard edges.

Moving the mouse into the canvas temporarily adds an eleventh orb. You are participating in the field too.

The other road, opposite to per-pixel coloring, is extracting isolines and stroking them. Marching squares looks up a table per grid cell from the four corner field values and connects them, turning the $f = 1$ isosurface into a real vector contour that stays sharp when scaled and can be used for collision detection. In 3D the counterpart is marching cubes, the general algorithm for extracting surfaces from scalar fields.

Per-pixel evaluation wins on speed, isoline extraction wins on precision, and that trade runs through the whole field of field rendering.

Plasma is the signature effect of the 1990s demoscene, its kernel the superposition of four sine fields with different phases:

The four terms lay down periods along the horizontal, vertical, diagonal and radial directions respectively. The fourth’s $\sqrt{x^2+y^2}$ sends concentric ripples spreading from the center of the picture.

The resulting $v \in [-4, 4]$ passes through a three-channel palette. Red, green and blue each take one phase of $\sin$ (offset from each other by $2\pi/3$), so the continuous variation of $v$ sweeps out a color wheel. “Sine palettes with staggered phases” is the universal trick of false-color rendering, used everywhere from terrain elevation maps to medical imaging.

The frequency slider (scale 0.03 to 0.3) adjusts the spatial period, and the speed slider adjusts how fast the time terms advance.

Lissajous applies the same “parametric evaluation” to a curve. $x = \sin(a u + \varphi)$, $y = \sin(b u)$, one frequency hung on each orthogonal direction.

The frequency ratio $a:b$ determines the figure’s lobe structure (1:1 is an ellipse, 3:2 a three-lobed figure), and the phase difference $\varphi$ flips the whole figure over like a twisting coin. Physicists later used it on oscilloscopes to measure frequency ratios; in essence it is “the orthogonal projection of two oscillations”.

A dividing line connected to rational numbers hides here too. When $a:b$ is rational the trajectory closes into a curve; when it is irrational it never closes and winds densely around the whole torus over time. Park the frequency slider on any non-integer ratio between 1.5 and 2 and let the animation run for a few minutes, and you can watch with your own eyes that “closed” and “dense” are separated by nothing but one rational rounding.

A noise field is a random function, a fractal a self-similar function, Voronoi and metaballs functions of distance and potential, Plasma and Lissajous combinations of sines. All of them are “implicit definition + per-pixel evaluation”, the same route as the per-pixel scanlines in Retro Signals and Pixel Rendering.

Thanks for reading! Follow me if you'd like~

Generative Systems

Fri May 22 2026

9536 words · 49 minutes

Algorithms & Data StructuresFrontend EngineeringAlgorithmsProbabilityGraphics

---

© MuxiaoWF | CC BY-NC-SA 4.0

Share

Share this article

X (Twitter)TelegramRedditFacebookEmailQQ

Close

Close