Skip to content

sync_to_async_iter's cleanup path blocks the event loop with a direct thread.join() #2373

Description

@AmirF194

Describe the bug

sync_to_async_iter's cleanup path calls a blocking thread.join(timeout=1.0) directly on the event loop thread instead of hopping it through an executor. When a consumer exits early or cancels (early break, timeout, aclose()), that finally block can stall the whole event loop for up to a second, not just the task doing the cleanup. This function backs the async file-listing step in the localfs, OCI object storage, and Google Drive source connectors, so a cancelled or timed-out listing against a slow directory tree or a slow storage API call stalls every other concurrent coroutine on the loop, not just the listing itself.

Compare with async_to_sync_iter's mirror-image thread.join(timeout=1.0): that one is fine, since that function asserts at entry that it's running from sync code with no event loop.

To Reproduce

Ran this against a fresh clone at edb59a99 (current main), Python 3.11, no extra deps besides typing_extensions:

import asyncio, time, threading, importlib.util

spec = importlib.util.spec_from_file_location(
    "m", "python/cocoindex/connectorkits/async_adapters.py"
)
m = importlib.util.module_from_spec(spec)
spec.loader.exec_module(m)

def slow_sync_iter():
    yield "first"
    time.sleep(5)
    yield "second"

async def main():
    ticks = []
    stop = threading.Event()

    async def heartbeat():
        while not stop.is_set():
            ticks.append(time.monotonic())
            await asyncio.sleep(0.05)

    hb = asyncio.create_task(heartbeat())
    agen = m.sync_to_async_iter(slow_sync_iter)
    await agen.__anext__()  # "first"

    t0 = time.monotonic()
    try:
        await asyncio.wait_for(agen.__anext__(), timeout=0.1)
    except asyncio.TimeoutError:
        pass
    await agen.aclose()  # runs the finally: block under test
    t1 = time.monotonic()
    ticks.append(t1)

    stop.set()
    hb.cancel()
    gaps = [b - a for a, b in zip(ticks, ticks[1:])]
    print(f"cleanup wall time: {t1 - t0:.3f}s, max heartbeat gap: {max(gaps):.3f}s")

asyncio.run(main())

Expected behavior

The heartbeat task should keep ticking every ~0.05s throughout, since it has nothing to do with the iterator being cleaned up.

Actual behavior

cleanup wall time: 1.102s, max heartbeat gap: 1.001s

The heartbeat stalls for the full duration of the thread.join(timeout=1.0) call. Patching that one line to await loop.run_in_executor(None, thread.join, 1.0) (matching how the rest of the function already treats thread interaction) brings the max gap back down to ~0.05s in the same repro.

CocoIndex Version

Reproduced against main at commit edb59a9930a8541eefa7ff20a2a3d8d8e6e43366.

Additional context

Fault is in python/cocoindex/connectorkits/async_adapters.py, sync_to_async_iter's finally block (the thread.join(timeout=1.0) call). Happy to send a PR for the executor-hop fix plus a regression test (slow sync iterator + cancellation + a concurrent heartbeat task asserting the loop stays responsive) if that's useful.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions