One durable storage change. Many optional live migrations.
Squash proves that a compact S layer has the same merged view as its raw L-layer run, then commits it to the active manifest. A post-commit sweep tries to release historical pins; failure in that sweep cannot undo the storage commit.
Transaction boundary
Commit first; reclaim only after verification.
Before
- L₄ · newest
- L₃
- L₂
- B₀ · base
Committed head
- L₄ · racing head kept
- S₅ · L₃ + L₂
- B₀ · base
Correctness
merged(B₀, L₂, L₃, L₄) = merged(B₀, S₅, L₄)
Reclamation
L₂ and L₃ disappear only after every lease releases them.
The expensive fold runs outside the writer lock.
A plan lease turns immutable source layers into stable build inputs. The short exclusive phase only revalidates, promotes, syncs, and swaps metadata.
- 01brief shared lock
Plan
Read the active manifest, collect each live lease’s newest layer as a boundary, partition maximal raw runs of at least two layers, and acquire a plan lease.
No base B* layer and no live boundary enters a block.
- 02no storage lock
Build
Fold each selected run newest-first into a nonce staging tree. Preserve whiteouts and opacity; hardlink regular-file winners and recreate symlinks.
Publishes may continue because the plan lease pins every source.
- 03exclusive lock
Commit
Re-read head, require each raw run to remain contiguous, rename staged S layers, sync the storage filesystem once, and atomically replace the manifest.
Record substitutions, then release the plan lease—the transaction’s only GC.
Race-safe head preservation
Because publishes prepend while flatten runs, commit re-reads the newest manifest and splices only still-contiguous planned runs. A publish that raced the build remains at the head.
Compaction work scales with metadata visited, not base-image copies.
These bounds describe the current implementation’s dominant work. L = active layers, R = live leases, E = visited entries, V = compacted output entries, P/T/F/M = process inspection inputs, N = sessions, W = sweep width.
| Stage | Time | Extra space | Why |
|---|---|---|---|
| Plan | O(L + R log R) | O(L + R) | Scan L active layers and order the newest boundaries from R live leases. |
| Flatten | O(E log E) worst case | O(V) entries | Walk E source entries and choose winners in ordered maps. Regular payload blocks are shared by hardlink; V is the compacted visible output. |
| Commit | manifest recheck + Q renames | O(L + Q) | Revalidate and splice Q blocks against the latest manifest; syncfs durability usually dominates wall time. |
| Lease rewrite | generation × chain scan | O(L) | Apply append-ordered substitutions to one leased chain, then verify every rewritten layer directory still exists. |
| Quiesce | O(P + T + F + M) | O(P + T) | Scan /proc, task threads, file descriptors, and memory-map rows before the switch. |
| Session sweep | O(N) work · ≈⌈N/W⌉ waves | O(N) | Remount N sessions with bounded width W. Current default W is 4; result ordering remains stable. |
Hardlinking is why regular file payloads are not copied into S. The new tree still needs directory entries, symlinks, whiteouts, opaque markers, xattrs, and staging space until commit.
A shorter chain is acquired before the old chain can move.
Each committed S layer records the raw run it replaces in an append-ordered, in-memory map. Rewriting applies older generations first so substitutions compose across later squashes.
Append order
Pin-overlap timeline
There is no unpinned instant.
- 1OLD lease holds raw layersOLDNEW
- 2Acquire NEW replacement leaseOLDNEW
- 3Quiesce and switch mountOLDNEW
- 4Verify NEW, then release OLDOLDNEW
Missing substitution
Identity; keep the existing lease.
Missing layer directory
Identity; never construct a partial chain.
No shorter chain
Identity; there is nothing to remount.
Stop every observer, then prove it holds no workspace pin.
The switch runs inside a per-session gate. Read errors are uncertainty, not permission: any child mount, escaped namespace, new task, cwd/root/fd/map pin, or unclassifiable state cleanly blocks migration before the point of no return.
- 01
Read holder mountinfo first. The workspace must be OverlayFS and have no child mounts.
- 02
Discover the union of tasks in the holder mount namespace and the session cgroup.
- 03
SIGSTOP every observable task, then poll every thread until stopped, exited, or safely traced.
- 04
Discover again. Any new task means membership changed, so the attempt stops.
- 05
Inspect every frozen thread’s mount namespace, cwd, root, file descriptors, and memory maps.
FrozenTasks is a guard
On clean skips and verified switches, dropping the guard sends SIGCONT. On a faulty post-PONR state, the guard is deliberately forgotten so no task observes a partial mount.
Interactive cwd is a real pin
A live shell whose current directory is inside the workspace predictably blocks remount. The session stays correct and its old lease remains authoritative.
We can estimate remount time—not yet the exact application pause.
Retained traces wrap the complete per-session remount operation. They establish an idle-session baseline, but do not isolate the interval from SIGSTOP to SIGCONT and are not a running-service availability SLA.
Observed remount span
145 successful measurements across 29 retained runs
- p50
- 5 ms
- p95
- 6 ms
- pooled p99
- 7.1 ms
- max
- 18 ms
What was measured
One live, idle session; one eight-layer squash block; 4 KiB per layer; and sweep width four. The pooled p99 is computed from the raw observations because individual reports do not publish p99.
What users may observe
SIGSTOP pauses application work without closing its sockets. An in-flight request may gain latency or hit its client timeout, but the session is not disconnected. Active-service pause percentiles still need a dedicated SIGSTOP-to-SIGCONT benchmark.
Safety ceilings are not latency targets
The default freeze handshake budget is 500 ms and the remount runner timeout is 30 s. Pre-PONR uncertainty resumes tasks; post-PONR uncertainty keeps them frozen until the session is destroyed rather than exposing a partially switched mount.
Stage NEW, move OLD aside, move NEW into place.
The runner enters the holder’s user and mount namespaces. It restores hidden-path masks before either mount move, keeps O_PATH descriptors across path changes, and never uses lazy detach.
Boot-time kernel gate
Live remount stays disabled until a same-upperdir, fresh-workdir probe—including whiteout and opaque-directory behavior—passes on the running kernel. An unprobed or failed gate degrades squash to commit-only maintenance.
- 01
Lift masks
Temporarily expose daemon-owned paths inside the holder mount namespace.
- 02
Prepare scratch points
Create staging, rollback, and a fresh sibling workdir under the session run directory.
- 03
Mount NEW
Reuse the session upperdir, supply compacted lowerdirs, and open O_PATH handles for staging, rollback, and OLD.
- 04
Restore + probe
Restore masks before any move, then prove the staged root is a readable OverlayFS mount.
- 05point of no returnpoint of no return
Move OLD → rollback
The first MS_MOVE detaches the visible workspace mount. Its success is the point of no return.
- 06
Move NEW → workspace
Reopen the underlying workspace path and move the staged overlay onto the public mountpoint.
- 07
Probe visible NEW
Verify the now-visible mount through its retained descriptor before any task resumes.
- 08
Strictly unmount OLD
Drop the OLD-root descriptor and unmount rollback without lazy detach. EBUSY is parked, not forced.
- 09
Report
Return first_move_succeeded, mount_verified, and a class:detail reason for deterministic classification.
Two booleans decide whether tasks may resume.
The parent classifies the runner report rather than inferring success from process exit. If the report disappears, the holder’s workspace mount ID is compared with the pre-switch value.
| Runner signal | Classification | Lease action | Tasks |
|---|---|---|---|
| first_move = false | Clean skip | Release NEW · keep OLD | Resume |
| true · verified = false | Faulty session | Park NEW until destroy | Keep frozen |
| true · true · switched | Migrated | Install NEW · release OLD | Resume |
| true · true · rollback EBUSY | Verified, parked | Install NEW · park OLD | Resume |
| true · true · other detail | Faulty session | Park NEW until destroy | Keep frozen |
| report missing | Compare mount IDs | Same = clean · else faulty | Resume only if unchanged |
Before manifest replace
The old manifest remains authoritative. In-process failures remove staged or promoted S directories.
After storage commit
Squash is durable even if every remount skips. Old layers remain leased until sessions release or migrate.
Daemon death
RAM-only substitutions and parked-lease state vanish. Holder PDEATHSIG and startup reaping remove dead sessions; manifest recovery stays fail-closed.
Autosquash is an internal scheduler, not another storage algorithm.
The architecture chapter’s zero-argument manual operation still exists. Current code additionally runs the same action from a one-worker background engine when the configured active-layer policy matches.
Product config
100 active layers
The shipped prd.yml opts into squash_at_n_layers: 100.
Schema default
disabled when omitted
A custom config with no policy creates no queue and no worker.
Match rule
active ≥ threshold
B, L, and prior S layers count. The configured minimum is 3.
Wake-up signals
startup + real commit
No-op publishes and S layers created by squash do not recursively notify.
Asynchronous convergence
Publishing never waits for compaction.
Notify
A real L-layer commit enqueues a cheap signal after audit and session-destroy ordering.
Coalesce + recheck
The effective capacity-one queue merges bursts. The worker reads live head, waits for the shared action gate, then reads again.
Reuse squash
The same plan, flatten, commit, remount sweep, telemetry, and failure semantics run in the background.
Threshold does not guarantee a block
Live lease boundaries can partition the manifest so no run of two raw layers is eligible. The worker records no_squashable_blocks, leaves head unchanged, and waits for a later real commit or restart—there is no hot retry loop.
Recover durable truth; reap runtime state that cannot still be alive.
The daemon does not attempt to resurrect workspace holders or reconstruct RAM-only leases. Parent-death signals make persisted sessions provably dead, while the active manifest remains the authority for storage cleanup.
- 01
Remove export spools
Delete the export-owned spool directory whose RAM registry vanished.
- 02
Probe kernel floor
Assert required kernel behavior and set the live-remount feature gate.
- 03
Reap sessions
Read manager.json, remove contained run dirs, and reset persisted handles.
- 04
Sweep storage
Keep active manifest layers; remove staging and unreferenced non-base artifacts.
What survives
The atomic active manifest and every referenced immutable layer remain durable. They are sufficient to reconstruct the current LayerStack view without a live workspace or lease registry.
What is intentionally lost
Live command registries, namespace holders, in-memory leases, substitutions, and parked remount state are not recovered. Their owning sessions are reaped; clients create fresh sessions from durable head.
Storage deletion fails closed
A missing, unreadable, empty, or pre-versioned manifest deletes nothing. With a valid keep-set, base-layer IDs are still never removed by the boot sweep.