feat(scroll): capture a scrolling region and stitch it (--scroll) - #69
Open
jondkinney wants to merge 4 commits into
Open
feat(scroll): capture a scrolling region and stitch it (--scroll)#69jondkinney wants to merge 4 commits into
jondkinney wants to merge 4 commits into
Conversation
A tall or wide capture (a stitched scroll, a pinned panorama) is crushed to an unreadable sliver by fit-to-window. This adds a view transform on top of the fit: a zoom multiplier (1 = whole image visible, up to ~4 screen px per source pixel) and a pan offset, folded into editImageRect() so annotations, hit-testing and rendering map through it unchanged. Chrome anchors to a separate fit-only baseImageRect(), and once zoomed it anchors above the viewport band so it never floats over the content. Gestures, chosen so scrolling never changes the zoom: - Ctrl+wheel, Ctrl+plus/minus zoom about the cursor, keeping that image pixel under it; Ctrl+0 fits. - A plain wheel scrolls a zoomed capture, Shift+wheel scrolls sideways, and both are inert at fit. A selected layer still scales and a drawing tool still adjusts its size, unchanged. - Bare + / - / 0 also zoom, so an oversized capture stays navigable from the keyboard alone. - Middle-drag pans. The pan is clamped so the image always covers the viewport where it is larger and stays centered where it is smaller. Wheel handling accepts pixelDelta as well as angleDelta, since high resolution touchpads may send only pixels, and ignores the zero-delta scroll begin/end markers. A discrete step follows whichever axis carried the notch: Alt+wheel arrives as a horizontal delta on some setups, and reading only the vertical one would leave every Alt-adjusted setting able to rise and never fall. main() also lifts QImageReader's 256 MB decode limit, without which a tall capture cannot be opened by --file or --pin at all. Smoke (exit 116): one horizontal notch each way lands back where it started. Smoke (exit 111): a 600x6000 capture through zoom, pan, middle-drag, Ctrl+0 restoring the fit exactly, scrolling being inert at fit, and a rectangle drawn while zoomed surviving the fit.
Scroll capture needs many frames per second of one output, so it cannot pay grim's process spawn or a fresh session handshake per frame. This adds an OutputCapture class that opens an ext-image-copy-capture session on a named output over its own Wayland connection and grabs repeatedly into the same shm buffer; captureOutput() is the one-shot wrapper. It reuses the surface-capture machinery: the registry also binds wl_output (v4, for the connector name) and the ext-output source manager, and the frame/session dispatch is factored into connectCaptureDisplay() and captureFrame(). Frames are captured without the cursor and returned upright through the existing transform normalisation. The session is defensive about the two ways it can go wrong in practice: - Buffer geometry and format are recorded when the buffer is created and used for attach, damage, copy and bufferSize(), because the compositor can re-announce constraints in the same dispatch batch as a frame's ready event and the live values would then describe a different mmap. A failed rebuild keeps the rebuild flag set rather than attaching a null wl_buffer, and a re-announcement clears the stale shm format list. - grab() takes a timeout (default 2 s) and sessionStopped() reports a compositor-ended session, so a caller can tell a dead session from a quiet screen. ext-image-copy-capture only delivers on damage, so a static output would otherwise be indistinguishable from a hang. Measured on Hyprland 0.56.2 (DP-3, 6144x3456 @2): 26-93 ms per frame. Smoke: wl_output and the output source manager added to the destruction order check; `OMASNAP_SMOKE_OUTPUT=<connector>` gates a live five-frame session against a real compositor.
The pure half of scroll capture: everything that decides how far a page moved between two frames and how to assemble them, over QImage with no compositor or windowing dependency, so all of it runs in the offscreen smoke suite. - classifier: GrayView downsample (cross-axis /4, motion axis exact), scoreShift (fixed-window MAE correlation, coarse then fine), stationary-edge cropping so viewport-fixed chrome does not floor every candidate, and a signed cascade giving Stationary / Forward / Reverse / Ambiguous / Unmatchable with error and confidence. - accumulator: StitchAccumulator with retained tail bands, per-depth error ledgers, the stationary trailing strip (window borders, fixed footers) and the near-stationary trailing zone (drop shadows), forward and reverse paths, and finish for either axis. - lookahead: a path resolves only when F0->F1, F1->F2 and F0->F2 agree cumulatively within 2 px and beat every competitor; a probe that did not move reports StationaryProbe rather than inventing a band from a periodic alias. - AutoCapture plus CaptureHandshake and AutoStepCalibration: the decision loop that pairs scroll ticks with frames, blocking the scroller until each frame is recorded so nothing scrolls past uncaptured. The downsampler reads whichever byte order it is handed. Frames arrive as RGBA8888 from the capture path, so accepting only the BGRA formats would copy every frame a second time before a single pixel was compared. A capture is bounded by memory (512 MB of RGBA), not just by INT_MAX extent. The retained bands already hold roughly the finished image, so an unbounded capture is not refused at allocation time on Linux, it is OOM-killed part-way through assembly, after a long scroll. Reaching the budget is a state rather than a failure: ManualCapture reports Full and AutoCapture halts with ReachedLimit, both leaving the capture intact for the caller to finish. Separately from that hard limit, exceedsWidelyOpenableEdge() reports when a capture passes 32767 px on an edge, which is where many renderers stop (they address pixels with signed 16-bit coordinates). Nothing breaks at that size here, so it is advisory: it sits at about 60% of the budget, capture continues past it, and the caller only mentions it. Vertical assembly writes rows straight into the image instead of filling a full-size vector and copying it, which halves peak memory and drops a pass (74 ms -> 34 ms on a 359 MB stitch); a reverse capture is emitted back-to-front rather than reversed afterwards. The horizontal path already assembled in place. stitch-replay is an offline harness: point it at a dumped frame directory and it classifies, stitches and writes the result without a compositor, which is also how a bad stitch gets debugged. Smoke (exit 109): scoreShift bit-identical to a naive reference across axes, shifts and steps; stationary-pair edge capping; a known shift classifying Forward and stitching to the expected height; synthetic lookahead fixtures through the candidate-set decisions; and AutoCapture fed through every state-machine row.
Drag a region, choose manual or automatic, scroll (or let it scroll), and the frames are stitched into one tall or wide capture that opens in the editor. The frame resizes by its corners and edges and moves by its border; the page underneath stays live throughout, so it can be scrolled into place before capturing and by hand during a manual capture. The capture overlay and this one wear the same chrome, badge, key guide, status line, and S and A swap between them. The region is remembered for the session (a line in the runtime dir, gone at reboot); R brings it back. An auto capture that stops short can be picked up again with Continue.
This was referenced Aug 19, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
omasnap --scrollcaptures a scrolling region and stitches the frames into one tall (or wide) image, which opens in the editor like any other capture.Drag a region, pick manual or automatic, scroll (or let it scroll), and it assembles. The frame resizes by its corners and edges and moves by its border. The page underneath stays live throughout, so you can scroll it into position first and by hand during a manual capture. The region is remembered for the session (one line in the runtime dir, gone at reboot) and
Rbrings it back; an automatic capture that stops short picks up again with Continue. This overlay and the capture overlay wear the same chrome, andS/Aswap between them.The three commits
None does anything alone, and the seams between them are where the interesting failures live.
feat(capture)addsOutputCapture, which opens anext-image-copy-capturesession on a named output over its own Wayland connection and grabs repeatedly into the same shm buffer, reusing the existing surface-capture machinery: scroll capture needs many frames a second and can't pay a process spawn or a handshake for each. Two deliberate bits of paranoia. Buffer geometry and format are recorded at buffer creation rather than read live, because the compositor can re-announce constraints in the same dispatch batch as a frame's ready event, and the live values then describe a different mmap. Andgrab()takes a timeout withsessionStopped()reporting a compositor-ended session, because the protocol only delivers on damage, so a static screen is otherwise indistinguishable from a hang. On Hyprland 0.56.2 (DP-3, 6144x3456 @2), 26 to 93 ms a frame.feat(stitch)is the pure half, overQImagewith no compositor dependency, so all of it runs offscreen in the smoke suite. A downsampled fixed-window MAE correlation with stationary-edge cropping (so viewport-fixed chrome doesn't floor every candidate) classifies each pair as Stationary, Forward, Reverse, Ambiguous or Unmatchable, and a path only resolves whenF0->F1,F1->F2andF0->F2agree cumulatively within 2 px and beat every competitor. That last rule is what stops a periodic background inventing a plausible shift. The accumulator handles retained tail bands, sticky headers and footers, and drop shadows. Captures are bounded by memory (512 MB of RGBA) rather than extent, because Linux overcommits: an unbounded capture isn't refused at allocation time, it's OOM-killed part-way through assembly after a long scroll. Hitting the budget is a state, not a failure.feat(scroll)is the overlay: region selection, the manual/automatic choice, the live page underneath, Continue, and the shared chrome.Bits I'd like your read on
The overlay chrome moved into
src/overlay-chrome.{hpp,cpp}and both overlays draw through it, which touches the existing capture overlay. That's the part most likely to disagree with where you'd want the code to live.Keeping the page interactive means holding the input region to the frame's chrome only and taking the keyboard grab only while the overlay needs it, because on Hyprland an exclusive keyboard grab also pins pointer focus, even over an input-region hole. That's why the grab is released before an automatic capture starts.
exceedsWidelyOpenableEdge()is advisory, not a limit: past 32767 px many renderers stop (signed 16-bit pixel coordinates), but nothing breaks at that size here, so capture continues and the editor mentions it once on opening.Testing
Exit 109 (
scoreShiftbit-identical to a naive reference, stationary-pair edge capping, a known shift stitching to the expected height), 108 (lookahead fixtures through the candidate-set decisions), 112 (AutoCapturethrough every row of the state machine), plus 116 and 111 from the PR under this one.stitch-replaystitches a dumped frame directory with no compositor, which is how I debug a bad stitch.I also keep a corpus of real frame sequences (~360 MB, five scenarios including sticky headers and a horizontal one) and gate every change on the stitch being pixel-identical to a recorded result. Too big to carry here, but I can put it somewhere you can run it.
test/all-featureson my fork is all nineteen branches merged on currentmain, green on the suite and that gate: https://github.com/jondkinney/omasnap/tree/test/all-features