Tuning Chalk Tool Performance: From Full-Screen Passes to Scissor Rectangles

Red and blue chalk on the device test build, 155 strokes at 255% zoom.

The chalk tool in Lulucat Notes had a particular performance failure: writing over a blank area felt smooth, but moving into an area already filled with chalk strokes left the pen tip behind. Carrying on in that same area gradually slowed canvas panning too.

A single page of ordinary handwriting was enough to reproduce it: 300% zoom, 70 visible chalk strokes in the local area, and 3,571 input sample points altogether. Blank areas remained fluid; the slowdown appeared only where the strokes were concentrated.

After the fix, the page can continue to receive new writing at 255% zoom, whilst existing strokes retain their full clarity during pen-down and panning.

The device test screenshot, 155 strokes in total. At this zoom level, existing strokes do not briefly change clarity during pen-down or panning.

Why chalk needs a scratch texture

An ordinary pen can composite each circular stamp directly onto the ink texture using source-over blending. Chalk adds a grain-gating layer: the renderer first accumulates body coverage and depth for an entire stroke, then uses a fixed grain texture to choose which positions receive chalk dust, and finally composites the result onto the existing ink.

This scratch texture isolates a single chalk stroke. That matters because stamps within one stroke overlap heavily; if every stamp were grain-gated separately, the stroke centreline would accumulate repeated colour and the chalk pores would shift with the sampling density.

At high zoom levels, Lulucat Notes redraws the vector strokes visible in the current viewport. The previous implementation carried out these steps for every visible chalk stroke:

1. End the main render encoder;
2. Clear the scratch texture;
3. Draw this chalk stroke into the scratch;
4. Reopen the main render encoder;
5. Composite the scratch back to the drawable with a full-screen triangle.

The semantics of an individual stroke were correct, but the work covered a far larger area than necessary. The iPad’s drawable was 2732×2048 — roughly 5.6 million pixels. Every chalk stroke triggered one scratch pass and one full-screen composite. 70 chalk strokes therefore meant approximately 141 render encoders and 70 full-screen composites.

Let denote the number of visible chalk strokes and the drawable pixel count. Looking only at work that scales with pixel coverage, the old implementation was close to

Each chalk stroke also incurred a fixed render-pass overhead, so this cost grew linearly with as well. The 3,571 input points contributed only a secondary cost. The part that scaled with the local stroke count was the full-screen work scope triggered by each stroke.

We took measurements on a 12.9-inch iPad Pro (5th generation, M1) running iPadOS 18.6.2. GPU timestamps from the same viewport were compared before and after the change, using command-buffer timestamps in the same Debug device build on this particular iPad — referred to below as LucasPad. These ranges are typical fluctuations in multi-frame logs, not a frame-rate commitment for a build distributed to users. At 70 visible chalk strokes, one frame typically required 52–60 ms of GPU time; in an area with roughly 120 strokes, GPU time rose to 77–80 ms.

Using the full-screen rectangle area of the scratch and composite passes, the theoretical work scope per frame grew from approximately 783 million pixels to 1.34 billion pixels. This figure is the sum of rectangle areas; it is not the same as fragment invocation counts, video-memory read/write bytes, or GPU hardware counters. Metal’s fast clear, attachment load/store, and pass switching remain under GPU and driver control.

This is also why blank areas stayed smooth. Visibility culling skips strokes outside the viewport; in a blank area is close to zero, whereas in a dense area keeps climbing.

A wrong answer at 0.85 ms

The app already had a full-page ink texture baked at two pixels per point. We tried displaying that texture directly during writing, panning, and zooming, retaining only the current Apple Pencil stroke as live vector; once the interaction ended, one additional frame would re-render the high-resolution vector result.

The approach was remarkably fast. In the same dense area at 300% zoom, GPU time fell to 0.84–0.85 ms and no longer increased with the number of existing chalk strokes.

On the actual device, the drawback was just as clear. At 300% zoom, roughly six screen pixels per point were required, but the cache provided only two. As soon as the Apple Pencil touched down, all existing strokes became a soft, low-resolution image; lifting the Pencil made them snap back to full clarity.

The tester had one response: “When I write, the whole canvas turns blurry. It clears when I let go.”

We removed the optimisation. 0.85 ms was the lowest measured result, but it did not make an acceptable chalk tool. Existing strokes are part of the writing feedback, so their clarity cannot change at pen-down.

Limiting each chalk stroke to its own rectangle

The implemented fix retained scratch and compositing per stroke, reducing only the pixel area each operation covered. Each stroke already had a canvas bounding box derived from the union of all its stamp radii. The renderer transforms this bounding box into the current viewport’s drawable coordinates and pads it by two pixels for an antialiasing margin:

That same scissor rectangle is used to clear the scratch, draw the stroke, and composite the result back to the main surface.

let rect = displayScissorRect(for: stroke.bounds, viewport: viewport)

scratchEncoder.setScissorRect(rect)
clearScratchExplicitly()
drawStrokeIntoScratch(stroke)

mainEncoder.setScissorRect(rect)
compositeChalkFromScratch(stroke)
mainEncoder.setScissorRect(fullDrawable)

The same logic is used for baking and partial replay on the 4096² ink texture, so the high-zoom display and the settled ink layer do not create two different chalk behaviours.

Two details are easy to miss.

First, a render pass’s loadAction = .clear takes place during the attachment load stage and is not constrained by the rasterisation scissor. Continuing to use it would still clear the entire scratch texture. The fixed pass uses .dontCare, then draws a clear_fragment within the scissor. That rectangle is subsequently written in full, and the composite reads only the same rectangle, so old attachment contents do not need to be loaded.

Second, the outer scissor must be restored when each chalk stroke’s composite has finished. If that state-restoration line is omitted, subsequent pens, images, or selections remain clipped by the bounds of the previous chalk stroke, appearing to be missing.

Chalk grain still samples from absolute canvas coordinates rather than local UVs within the rectangle. Moving the scissor changes only which pixels the GPU processes; it does not change which grain-texture location a pixel reads. Adjacent rectangles therefore produce no texture seams, and dragging the canvas does not make the grain drift.

Considering only the pixel workload, the new work scope is close to

where is the axis-aligned bounding-box area of the -th chalk stroke on the current screen. The number of render encoders has not fallen, but each clear and composite is now bounded by the stroke’s screen bounding box.

Why same-colour chalk strokes were not batched

Most chalk strokes on the page share the same colour and density. It is tempting to draw dozens of strokes into the scratch in one go and composite just once. That would reduce render passes further, but it would change the colour and grain semantics in overlapping areas.

Consider a deliberately simplified case: two strokes share a grain-gate value at a given pixel, with body coverages and . In the actual shader, the gate also depends on each stroke’s pressure depth; this simpler case is enough to show that batching is not generally equivalent. The current per-stroke compositing produces

whereas merging the bodies first and applying one gate produces

The difference is . Whenever two strokes overlap and the grain gate is neither pure zero nor pure one, the results differ. Direct batching would alter how chalk dust lands at crossings.

Exact batching requires proof that the stroke pixels are mutually disjoint, or an independent atlas region for each stroke followed by compositing in the original order. Device acceptance testing kept the scissor approach, so this round introduced neither an atlas nor the complexity of managing one.

From a billion pixels back to a few million

The device test measurements were:

Scenario	Before fix	Precise scissor
70 visible chalk strokes, 300% zoom, writing	GPU 52–60 ms	≈ 9–10 ms
≈ 121 visible chalk strokes, 300% zoom	GPU 77–80 ms	13.7–15.6 ms
Theoretical rectangle scope per frame (scratch + composite)	783 M–1.34 B pixels	≈ 1.7 M–3 M pixels

We also verified the clipping bounds using 3,452 strokes and 202,710 sample points from a device document. At 0.5×, 1×, 2×, 3×, 5×, and 8× zoom, 186,408 viewport cases were generated; every point sprite that could produce non-zero coverage fell within the computed scissor. The check covered canvas edges, viewport edges, and several offset combinations.

The chosen implementation does not switch to a low-resolution LOD according to interaction state. Low zoom levels still display the full-page ink texture; high zoom levels still redraw visible strokes as vectors. On the same side of the threshold, pen-down and panning do not replace existing strokes with a different level of clarity. During high-zoom vector redraw, the scratch clear and composite for each chalk stroke cover only its own screen bounding box.

GPU time did not capture clarity

GPU performance problems do not necessarily scale with the most visible quantity in the data structure. Here, 3,571 input points were an easy suspect; frame time was determined by the full-screen work triggered by each of the 70 chalk strokes, together with render-pass switching.

Visual semantics also limited the available optimisations. Per-stroke scratch, the original compositing order, and absolute canvas grain coordinates could not simply be removed. Same colour and same density only mean that the parameters match — they do not prove that overlapping results can be merged.

Feedback from the real device rejected the version with the lowest GPU time. The comment “the whole canvas turns blurry” supplied the product constraint that measurement alone had not expressed: when the Apple Pencil touches down, users are also observing the existing strokes.

The chosen implementation introduces no new interaction cache layer and does not reduce clarity. High-zoom vector redraw simply limits each chalk stroke’s work to its own screen bounding box. After it was loaded onto LucasPad again, the feedback was: “It looks excellent.”