WAL pipeline mode 6 — end-to-end scheme

WAL pipeline mode 6 — Stage A async transport, end to end

One commit's journey from XLogInsert to the ack, through the pipeliner's submit/reap split, the shared-memory rings, the pfsd event loop, and io_uring. Strictly ordered (Stage A): one device I/O in flight, completions arrive in submission order, and only a datasync completion proves durability.

BACKENDS · N PROCESSES POSTGRES SHARED MEMORY · ONE REGISTERED ZERO-COPY MEMFD SEGMENT WAL PIPELINER · 1 PROCESS, 4 THREADS · polar_wal_pipeline_mode = 6 WAL CONTEXT · SPSC SHM RINGS + EVENTFDS · CREATED PRE-FORK BY POSTMASTER PFSD DAEMON · WalCtxSrv EVENT-LOOP THREAD (ONE PER CONNECTION) KERNEL · IO_URING · BLOCK DEVICE backend: XLogInsert() writes record into WAL buffer page links range in recent-written array backend: commit → polar_wal_pipeline_commit_wait() bumps LogwrtRqst.Write / .Flush under info_lck wakes advance + write workers · waits on flush-event slot backend: commit acked woken when logFlushResult ≥ its LSN — durability proven only by a datasync completion WAL buffers (XLogCtl→pages) 8k pages · inside the registered zc segment page reuse gated on logWriteResult, which now advances on completion bounce slots depth × 8k, IO-aligned, in the zc segment · tail-page snapshots 1 : 1 with descriptors XLogCtl watermarks LogwrtRqst.Write/.Flush · ready_write_lsn logWriteResult / logFlushResult = COMPLETED watermarks (reaper-only writers) flush-event slots per-LSN-range mutex+cond backends sleep here, notify workers signal them advance worker recent-written array → advances ready_write_lsn, wakes write worker write worker (main thread) — SUBMITS, never blocks on I/O polar_wal_pipeline_async_write → XLogWrite (structure unchanged) local LogwrtResult = SUBMITTED watermark (never re-read from shared) · full pages → submit zero-copy straight from WAL buffers · partial tail page → memcpy 8k snapshot into bounce slot, submit slot · WriteRqst.Flush reached → submit datasync (close_fd on segment end) descriptor ring (local): cookie=tail → {kind, end_lsn, len, fd, seg_no} doorbell eventfd after every submit · EBUSY-proof: waits on tail − head < depth (freed by reaper) reaper thread (flush-worker slot) poll(completion efd, 10ms) → reap batch cookie must equal head — strictly in order write cqe → logWriteResult = end_lsn datasync cqe → Rqst.Flush ↑ · barrier · logFlushResult = end_lsn · wake walsenders segment end → archive notify · checkpoint request · close old fd any error → PANIC (failed WAL write) notify workers scan flush-event slots < logFlushResult, signal backends submit ring (SQ) · WalSqe 64 B cookie · op · ino · buf_id · buf_off · file_off · len SPSC: write worker → daemon loop doorbell eventfd client → daemon level-triggered epoll completion ring (CQ) · WalCqe 16 B + efd cookie · res — SPSC: daemon → reaper outstanding ≤ capacity ⇒ push can never overflow created in POSTMASTER (SDK control plane doesn't survive fork) · crash-restart → EEXIST → sync fallback WalCtxSrv loop — epoll { doorbell · io_uring CQ efd · stop } pop WalSqe → resolve under inode lock: · mount/inode refs held for the op's lifetime · pfsd_wal_resolve_pwrite: block map → (raw device fd, bda) · buffer table: (buf_id, buf_off) → daemon's mapping of the memfd fast-path contract: sector-aligned · single 4M block · no extension · block fully materialized — miss → SYNCHRONOUS slow path, order kept Stage A: at most ONE op on the ring · datasync submitted only after the previous completion is reaped — never IOSQE_IO_DRAIN walring — private io_uring wrapper slot per SQE: FREE → INFLIGHT → FREE on reap; generation stamped in user_data — a stale or misrouted CQE is detected, never misattributed no IOSQE_IO_LINK / IO_DRAIN, plain SQEs registered eventfd ticks per CQE fault injection for the crash harness: drop-nth-write · EIO-nth-write · crash-after-n io_uring: pwrite(raw fd, ptr, len, bda) O_DIRECT raw device fd (dop_raw_fd) — DMA reads the bytes directly from the shared memfd pages: no copy anywhere bda = blkno × 4M + offset, from the PFS block map datasync: IORING_FSYNC_DATASYNC(raw fd) device-wide flush — one datasync covers every segment on the pbd (no per-file fsync needed) /dev/loop101 (pbd) PFS blocks of the WAL segment files, fully materialized (no holes) 1 2 wake advance + write wait on slot 3 ready_write_lsn ↑ 4 ptr → (buf_id, off) tail 8k memcpy 5 doorbell++ 6 7 8 fsync alone, gated on prior reap zero-copy: device DMA reads the same physical pages the backends wrote 9 WalCqe + efd++ 10 11 results published here only 12 LEGEND submission path (WalSqe: pwrite / datasync) completion path (WalCqe → watermarks → ack) data (zero-copy pages / bounce snapshot) control / wakeups DEGRADED PATHS · transport unavailable at postmaster init (no pfsd, zero-copy off, daemon without CAP_WAL_ASYNC, EEXIST after crash-restart) → mode 6 runs the synchronous mode-3 execution path; thread layout is identical, so backend wakeup routing stays valid · fast-path contract miss in the daemon (segment extension, unaligned, hole, 4M-boundary crossing) → that one op executes synchronously through pfsd_pwrite_svr in the loop thread; submission order is preserved · any transport or I/O error → PANIC: a failed WAL write can never be reported as durable
1

Backend XLogInsert writes the record into a WAL buffer page — physical pages of the one registered zero-copy memfd segment.

2

On commit the backend bumps LogwrtRqst, wakes the advance and write workers, and sleeps on its per-LSN flush-event slot.

3

The advance worker folds the recent-written array into ready_write_lsn, the writer's contiguity gate.

4

The write worker runs XLogWrite with its local LogwrtResult as the submitted watermark: full pages translate pointer→(buf_id, off); the live tail page is snapshotted (8k memcpy) into a bounce slot.

5

Each op becomes a 64-byte WalSqe pushed to the submit ring, with a descriptor (cookie → end_lsn, kind, fd) kept locally; the doorbell eventfd rings per submit.

6

The daemon's per-connection event loop wakes, pops the sqe, takes mount/inode refs, resolves the block map to (raw device fd, bda), and the buffer table to the data pointer.

7

The op goes onto the private io_uring via walring — plain SQE, slot-tracked, generation-tagged. Stage A keeps at most one in flight; a datasync launches only after the previous completion was reaped.

8

The kernel DMAs the bytes straight out of the shared memfd pages to the device at bda; a datasync is a device-wide FDATASYNC on the raw fd, covering every segment.

9

The CQE returns through walring (cookie/generation verified), becomes a 16-byte WalCqe in the completion ring, and ticks the completion eventfd.

10

The reaper thread reaps strictly in order — the cookie must equal the descriptor head, or PANIC.

11

Write completions advance logWriteResult; datasync completions advance logFlushResult (with the Write→Flush barrier), run segment-close actions, and wake walsenders + notify workers. The reaper is the only publisher of both results in mode 6.

12

A notify worker signals every flush-event slot below logFlushResult; the backend wakes with its commit durably acked.

INVARIANTS THE SCHEME RESTS ON

Submitted ≠ completed. The pipeliner's local LogwrtResult tracks what was handed to the transport; the shared logWriteResult/logFlushResult track what completed. Nothing outside the reaper moves the shared results.

Only datasync proves durability. A pwrite completion means the device accepted the write; acks, walsender advertisement, and the flush watermark all key off datasync completions, which cover every earlier op by strict ordering.

Buffer stability. WAL buffer pages aren't recycled until logWriteResult passes them — which now means write completed, so zero-copy reads never race recycling. The mutable tail page is snapshotted, with stock-pwrite semantics for racy bytes past the request end.

Flow control closes the loop. Outstanding ops are bounded by ring capacity on the client side, which is exactly what makes the daemon's completion push infallible and the descriptor ring 1:1 with bounce slots.