Rebuilding the Lulucat Notes Rendering Pipeline using Kimi K3

Last week, dragging a lasso selection across Lulucat Notes produced a visible wave: some screen tiles showed the selection at its new position while others still showed the old one, in the same frame. We replaced the whole rendering pipeline — Core Graphics bitmap plus CATiledLayer — with Metal, in six small steps, each verified on a real iPad before the next began.

The code was pair-programmed with Kimi K3, Moonshot’s open model, running on Fireworks. The human drove, decided, and tested; the model wrote nearly every line.

The pipeline

The old pipeline had two kinds of drawing that slowly drifted apart: strokes were baked into a bitmap for cheap display, and re-drawn as vectors whenever tiles needed more detail. The new pipeline has exactly one idea: everything ink-like is a point sprite. A pen stroke, a highlighter sweep, and an eraser dab are the same 32-byte vertex — position, diameter, colour — drawn by the same shader pair as GPU-rasterised circles spaced one point apart along the stroke’s arc length.

At one-point spacing a chain of circles deviates from a mathematically perfect capsule by about 0.075 points — a fifth of a pixel at our canvas density. In exchange, three tools collapse into one code path, and the GPU does what it is best at.

Around that idea the architecture is simple. Committed ink lives in a single 4096² texture. The UIScrollView survives, demoted to a pure gesture engine: its contentOffset and zoomScale feed a viewport uniform every frame, so panning and zooming write nothing. Each frame is five draws:

flowchart TB
    subgraph frame["Every frame: five draws"]
        direction TB
        paper["1 · paper blit"] --> ink["2 · committed ink"]
        ink --> live["3 · live stroke"]
        live --> sel["4 · selection"]
        sel --> dash["5 · lasso dashes"]
    end
    commit["stroke commit<br/>append stamps"] --> tex[("ink texture<br/>4096² render target")]
    replay["regional replay<br/>erase · delete · move · undo"] --> tex
    tex -. "sampled or re-drawn" .-> ink

Edits write the texture directly. Committing a stroke appends its stamps. Erasing, deleting, moving, and undoing replay the affected region behind a scissor rectangle: clear the region, re-draw the strokes that intersect it, done. Partial erasure keeps the ownership semantics from the previous post — an erasure belongs to the stroke it removes ink from — by drawing each erased stroke into a scratch texture, subtracting its own erasure paths with destination-out blending (), and compositing the result back. The scratch isolation stops an eraser from biting through paper or neighbouring strokes.

The selection that started all this is now drawn as point sprites too. Dragging it updates one uniform offset. Zero texture writes, zero tile invalidations — the wave is structurally gone, not mitigated.

The gatekeeper: a pixel diff

We did not delete the Core Graphics renderer. We demoted it to an offline reference implementation, and every Metal change must pass a pixel comparison against it on real stroke data captured on-device. The acceptance criterion is not “identical pixels” — two correct rasterisers legitimately disagree by a few grey levels along anti-aliased edges. The gate is structural: no missing ink, no offset, no colour drift, and no large difference anywhere away from ink.

This harness caught four of the five bugs we hit while building the offline renderer, all diagnosed by cropping the output to pixel level: a Swift/Metal struct stride mismatch (28 bytes versus 32, because Metal aligns float4 to 16 — the screen filled with colour blocks), [[point_size]] being unreadable as a varying in the fragment shader (every stamp came out square), two render encoders coexisting on one command buffer (everything black), and a missing start dab that left the first millimetre of quick strokes invisible.

Six steps, not one rewrite

The migration plan was six independently shippable steps: offline renderer passing the pixel diff; display shell with zero visual change; live stroke on GPU; selection on GPU; mutations writing the texture directly; high-zoom vector re-drawing (folded into step two, because “zero visual change” demanded it). Each step ended with a human — not a simulator, not a screenshot diff — writing, erasing, zooming, and dragging on the iPad Pro on the desk.

The device caught three bugs that every automated check missed. Above 100% zoom, strokes were drawn twice — soft texture underneath, sharp sprites on top — which read as a faint blur the human noticed in seconds. Committing a stroke flickered for one frame because the old overlay cross-faded out of sync with the texture update. And above 170% zoom, every note vanished: the visibility-culling rectangle used contentOffset in its scaled coordinate space, so it drifted away from the strokes as you zoomed. All three were one-line-to-one-function fixes, and none of them existed in any test we could have written beforehand, because we did not know to look for them. For a UI-first consumer app, that is why the human stays in the loop.

What Kimi K3 is like to work with

Fast, first of all. The loop of “discuss, write, build, install, look” ran in minutes, and a model that answers quickly changes how many loops you can afford in a day.

Second, it does not over-engineer. This codebase runs on explicit house rules — no backward-compatibility scaffolding before launch, complexity only when a device proves it necessary — and K3 follows them without being reminded. It did not add spatial indexes “for later,” did not wrap every call in defensive checks, did not speculatively abstract. Prompting it feels like working with a competent colleague who has read the house rules and actually believes them.

Third, give it tools and it uses them eagerly. We wired up image utilities — view, crop to a pixel region, resize — and the model started proactively cropping its own renderer output to diagnose the five harness bugs above. Having the tool reminded it to look.

The other half: K3 wrote most of the bugs in this story, including the coordinate-space one that made notes vanish. Its limits are real. What made the work safe was never the model being right; it was the harness catching rendering drift and the human catching feel. And yet, day to day, I could not reliably tell it apart from the frontier closed models we also use — Opus-class systems. On some axes it was plainly better: faster, and far less inclined to pad the codebase with defensive design.

How we want to build from now on

We are done with big-spec agentic development — the style where you hand a model a large specification and accept whatever lands. The failure mode is not bad code; it is code nobody understands.

What worked here, and what we will keep: small steps, each discussed before it starts, each understood by the human before it is built, each verified on the device it will live on. The model’s job is to be quick, precise, and honest about uncertainty. The human’s job is judgement, taste, and e2e verification — especially for UI-first consumer software, where the spec cannot describe what “right” feels like. Kimi K3 on Fireworks turns out to be well shaped for exactly this loop: fast enough to keep the loop tight, smart enough to keep the steps small and clean.

The wave is gone, the pipeline is one idea instead of two, and the process that got us there is staying.