Skip to content

[Bugfix][KV Offload] Decouple shared-region creator ownership - #53073

Open
Alex-ai-future wants to merge 2 commits into
vllm-project:mainfrom
Alex-ai-future:fix/kv-offload-shared-region-owner
Open

[Bugfix][KV Offload] Decouple shared-region creator ownership#53073
Alex-ai-future wants to merge 2 commits into
vllm-project:mainfrom
Alex-ai-future:fix/kv-offload-shared-region-owner

Conversation

@Alex-ai-future

Copy link
Copy Markdown
Contributor

Purpose

This PR separates the transient shared-region initializer from the lifetime
owner of singleton operations.

O_CREAT | O_EXCL elects a process only for the initialization branch that
checks /dev/shm capacity and calls ftruncate(). Calling that process the
"creator" is useful within that branch, but it is not a durable ownership
concept: it is an incidental result of startup timing, may belong to a process
that has already exited, and says nothing about which live worker should run a
later one-time action.

This becomes a concrete failure mode when the initializer exits after the file
has reached the expected size but before the other workers finish their
lifetime. The remaining workers can open and map the region successfully, yet
every live object may be a joiner with _creator=False. Any later singleton
operation gated on _creator, including cleanup unlink, then has no executor
at all and can leave the /dev/shm path behind. The same structural problem
applies whenever a future one-time operation is tied to this historical,
unobservable initializer identity.

Persisting _creator therefore gives cleanup and future singleton operations
an unnecessary and unreliable restriction. This PR removes that lifecycle
state and makes SharedOffloadRegion resolve singleton ownership from the
actual node-local rank, so a live, deterministic rank-0 worker can perform
singleton actions regardless of which process won O_EXCL.

The existing O_EXCL / ftruncate() / joiner polling protocol is intentionally
unchanged. In particular, this PR does not address creator-crash recovery,
stale-file takeover, orphan sweeping, or map-after-barrier unlink timing.

This is not a duplicate of #51317 or #52596: those PRs change the shared-memory
cleanup protocol, but both retain _creator as the post-initialization unlink
gate. This PR only replaces that ownership model.

AI assistance was used to prepare this change. The human submitter must review
every changed line and be able to explain the initialization and cleanup
semantics before submitting.

Test Plan

  • Run the focused CPU-only shared-region and KV-offload factory tests.
  • Verify that O_EXCL initialization and joiner waiting remain unchanged.
  • Verify that a non-owner can initialize the file, exit, and leave a joiner
    selected by local rank to perform singleton cleanup.
  • Verify that local rank, rather than layout slot rank or the O_EXCL winner,
    determines singleton cleanup.

Test Result

Ran on Linux:

.venv/bin/python -m pytest tests/v1/kv_offload/cpu/test_shared_offload_region.py -v
36 passed in 16.45s

PyTorch emitted 14 deprecation warnings. There were no test failures or skips.
No model evaluation is required because this change does not affect model
outputs, accuracy, or serving behavior.

Co-authored-by: Codex <noreply@openai.com>
Signed-off-by: Alex <jihui.huang@daocloud.io>

@claude claude Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Claude Code Review

This pull request is from a fork — automated review is disabled. A repository maintainer can comment @claude review to run a one-time review.

@mergify mergify Bot added the bug Something isn't working label Aug 20, 2026
@Alex-ai-future

Copy link
Copy Markdown
Contributor Author

@Etelis @orozery PTAL
I found this while working on the flock-based recovery for the stale-file restart loop reported in #51579, following @njhill suggestion on #51317 (#51317 (review)). The recovery replaces O_EXCL election and size polling with serialized preparation; tracing that flow exposed _creator being reused as a lifetime owner. I split that concern into #53073 and will continue the recovery work separately.

Signed-off-by: Alex <jihui.huang@daocloud.io>
Co-authored-by: Codex <noreply@openai.com>
@Alex-ai-future
Alex-ai-future force-pushed the fix/kv-offload-shared-region-owner branch from e58704e to b445ce7 Compare August 20, 2026 08:30
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

bug Something isn't working

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant