Chalk Tool Performance: From Full-Screen Passes to Scissor Rectangles

Red and blue chalk on the final device build, 155 strokes at 255% zoom.

The chalk tool in Lulucat Notes had a specific kind of performance problem: writing on a blank area felt smooth, but moving into an area already filled with chalk strokes made the pen tip fall behind. Continuing to write in that same area gradually slowed down canvas panning as well.

A single page of ordinary handwriting was enough to trigger it: 300% zoom, 70 visible chalk strokes in the local area, totaling 3,571 input sample points. Blank areas remained fluid; only the area where those strokes were concentrated became slow.

After the fix, the same page can continue to receive new writing at 255% zoom, and existing strokes retain their full clarity during both pen-down and panning.

The final device screenshot, 155 strokes total. At this zoom level, neither pen-down nor panning temporarily switches the clarity of existing strokes.

Why chalk needs a scratch texture

An ordinary pen can composite each circular stamp directly onto the ink texture with source-over blending. Chalk adds a grain-gating layer: the renderer first accumulates body coverage and depth for an entire stroke, then uses a fixed grain texture to determine which positions receive chalk dust, and finally composites the result onto the existing ink.

This scratch texture isolates a single chalk stroke. Isolation matters because stamps within the same stroke overlap heavily; if each stamp were individually grain-gated, the stroke centerline would accumulate repeated color, and the chalk pores would shift with sampling density.

At high zoom levels, Lulucat Notes redraws the vector strokes visible in the current viewport. The old implementation performed these steps for each visible chalk stroke:

1. End the main render encoder;
2. Clear the scratch texture;
3. Draw this single chalk stroke into the scratch;
4. Reopen the main render encoder;
5. Composite the scratch back to the drawable with a full-screen triangle.

The semantics of one stroke were correct, but the scope of work was far larger. The iPad’s drawable was 2732×2048 — roughly 5.6 million pixels. Every chalk stroke triggered one scratch pass and one full-screen composite. Seventy chalk strokes meant approximately 141 render encoders and 70 full-screen composites.

Let the number of visible chalk strokes be and the drawable pixel count be . Considering only the work that scales with pixel coverage, the old implementation was close to

Each chalk stroke also carried a fixed render-pass overhead, so that cost, too, scaled linearly with . The 3,571 input points contributed only a secondary cost. What scaled with the local stroke count was the full-screen work scope triggered by each stroke.

Measurements were taken on a 12.9-inch iPad Pro (5th generation, M1) running iPadOS 18.6.2. We compared GPU timestamps from the same viewport before and after the change, using command-buffer timestamps in the same Debug device build on this particular iPad — referred to below as LucasPad. The ranges below are typical fluctuations from multi-frame logs, not shipping frame-rate commitments. At 70 visible chalk strokes, a single frame typically required 52–60 ms of GPU time; in an area with roughly 120 strokes, GPU time rose to 77–80 ms.

Estimating by the full-screen rectangle area of the scratch and composite passes, the theoretical work scope per frame grew from approximately 783 million pixels to 1.34 billion pixels. This figure is the sum of rectangle areas and is not equivalent to fragment invocation counts, video-memory read/write bytes, or GPU hardware counters. Metal’s fast clear, attachment load/store, and pass switching remain under GPU and driver control.

This also explains why blank areas stayed smooth. Visibility culling skips strokes outside the viewport; in a blank area is close to zero, while in a dense area keeps climbing.

A wrong answer at 0.85 ms

The app already had a full-page ink texture baked at two pixels per point. We tried displaying this texture directly during writing, panning, and zooming, keeping only the current Apple Pencil stroke as live vector; after the interaction ended, one additional frame would re-render the high-resolution vector result.

This approach performed very well. In the same dense area at 300% zoom, GPU time dropped to 0.84–0.85 ms and no longer grew with the number of existing chalk strokes.

The problem on the actual device was equally clear. At 300% zoom, roughly six screen pixels per point were needed, but the cache provided only two. The moment the Apple Pencil touched down, all existing strokes turned into a soft, low-resolution image; lifting the Pencil, they snapped back to full clarity.

The tester said one thing: “When I’m writing, the whole canvas goes blurry. It clears up once I let go.”

The optimization was removed. 0.85 ms was the lowest measured result, but it was not an acceptable chalk tool. Existing strokes are part of the writing feedback; their clarity cannot change at pen-down.

Limiting each chalk stroke to its own rectangle

The final fix preserved per-stroke scratch and per-stroke compositing, and only reduced their pixel work scope. Each stroke already had a canvas bounding box derived from the union of all its stamp radii. The renderer transforms this bounding box into the current viewport’s drawable coordinates and pads it by two pixels for antialiasing margin:

The same scissor rectangle is then used for three things: clearing the scratch, drawing the stroke, and compositing the result back to the main surface.

let rect = displayScissorRect(for: stroke.bounds, viewport: viewport)

scratchEncoder.setScissorRect(rect)
clearScratchExplicitly()
drawStrokeIntoScratch(stroke)

mainEncoder.setScissorRect(rect)
compositeChalkFromScratch(stroke)
mainEncoder.setScissorRect(fullDrawable)

The same logic is also used for baking and partial replay on the 4096² ink texture, so the high-zoom display and the settled ink layer do not produce two different chalk behaviors.

Two details are easy to miss here.

First, a render pass’s loadAction = .clear occurs during the attachment load stage and is not constrained by the rasterization scissor. Continuing to use it would still clear the entire scratch texture. The fixed pass uses .dontCare, then draws a clear_fragment within the scissor. This rectangle is subsequently written in full, and the composite reads only the same rectangle, so old attachment contents do not need to be loaded.

Second, after each chalk stroke’s composite finishes, the outer scissor must be restored. If this line of state restoration is omitted, subsequent pens, images, or selections will continue to be clipped by the previous chalk stroke’s bounds, appearing as missing strokes or missing images.

Chalk grain still samples from absolute canvas coordinates rather than local UVs within the rectangle. Moving the scissor only changes which pixels the GPU processes; it does not change which grain texture location any pixel reads from. Adjacent rectangles therefore produce no texture seams, and dragging the canvas does not cause the grain to drift.

Considering only the pixel workload, the new work scope is close to

where is the axis-aligned bounding-box area of the -th chalk stroke on the current screen. The number of render encoders has not decreased, but each clear and composite is now bounded by the stroke’s screen bounding box.

Why same-color chalk strokes were not batched

Most chalk strokes on the page share the same color and density, and it is tempting to draw dozens of strokes into the scratch at once and composite only once. This would further reduce render passes, but it changes the color and grain semantics at overlapping regions.

Consider a deliberately simplified case: two strokes share the same grain-gate value at a given pixel, with body coverages and . In the actual shader, the gate also depends on each stroke’s pressure depth; this simpler case is enough to show that batching is not generally equivalent. The current per-stroke compositing produces

while merging the bodies first and then applying a single gate produces

The difference is . Whenever two strokes overlap and the grain gate is neither pure zero nor pure one, the results differ. Batching directly would alter how chalk dust lands at crossings.

Exact batching requires proving that the stroke pixels are mutually disjoint, or allocating an independent atlas region for each stroke and compositing in the original order. The final device acceptance retained the scissor approach, so this round did not introduce an atlas or its management complexity.

From a billion pixels back to a few million

Measurements from the final device test:

Scenario	Before fix	Precise scissor
70 visible chalk strokes, 300% zoom, writing	GPU 52–60 ms	≈ 9–10 ms
≈ 121 visible chalk strokes, 300% zoom	GPU 77–80 ms	13.7–15.6 ms
Theoretical rectangle scope per frame (scratch + composite)	783 M–1.34 B pixels	≈ 1.7 M–3 M pixels

We also verified the clipping bounds using 3,452 strokes and 202,710 sample points from a device document. At 0.5×, 1×, 2×, 3×, 5×, and 8× zoom, 186,408 viewport cases were generated; every point sprite that could produce non-zero coverage fell within the computed scissor. This check covered canvas edges, viewport edges, and various offset combinations.

The code under test does not switch to a low-resolution LOD based on interaction state. Low zoom levels still display the full-page ink texture; high zoom levels still redraw visible strokes as vectors. On the same side of the threshold, pen-down and panning do not replace existing strokes with a different level of clarity. During high-zoom vector redraw, each chalk stroke’s scratch clear and composite cover only its own screen bounding box.

GPU time did not capture clarity

GPU performance problems do not necessarily scale with the most visible quantity in the data structure. In this case, 3,571 input points were an easy suspect; what determined frame time was the full-screen work triggered by each of the 70 chalk strokes, along with their render-pass switching.

Visual semantics also constrained the available optimizations. Per-stroke scratch, original compositing order, and absolute canvas grain coordinates could not be removed at will. Same color and same density only mean the parameters match — they do not prove that overlapping results can be merged.

Real-device feedback rejected the version with the lowest GPU time. The remark “the whole canvas goes blurry” supplied the product constraint that measurement alone had not expressed: when the Apple Pencil touches down, users are also observing the existing strokes.

The accepted implementation introduces no new interaction cache layer and does not reduce clarity. High-zoom vector redraw simply limits each chalk stroke’s work to its own screen bounding box. After the test build was loaded onto LucasPad again, the feedback became: “Looks great.”