Skip to content

Conversation

@vladlearns
Copy link

@vladlearns vladlearns commented Jan 8, 2026

FIXES:
#1142
#1143

Bugs sum up

  1. On linux, default multiprocessing method is fork(). CUDA ctxs can't be inherited by forked child - when multiple workers try to init CUDA after forking, they crash. Only one worker survives /w uv retries;
  2. Got an issues running /w workers > 1 in 1 container, each worker starts as a separate process and imports the module fresh - cli args parsed in __main__ only exist in the parent process, so workers would miss that config and fall back to defaults/break.

What I changed

  • Set multiprocessing to spawn before starting uv, so each worker starts fresh = no fork = no CUDA init crashes;
  • Parent now dumps parsed args to json and stores them in var before starting uv;
  • Workers calls the factory, it reads the var, loads the json, and rebuilds the namespace, so every worker gets the same config.

QA

  • Solo-worker - same;
  • Multi-workers - config is passed to all workers;
  • Tested in Docker with 3 workers on 4090.

@vladlearns vladlearns changed the title fix: pass cli args to uv workers via env vars fix: support uvicorn multi-worker Jan 8, 2026
@vladlearns vladlearns changed the title fix: support uvicorn multi-worker fix(server): make uvicorn workers>1 work (factory + spawn for CUDA) Jan 9, 2026
@vladlearns
Copy link
Author

@leng-yue , @PoTaTo-Mika , @Stardust-minus , @Whale-Dolphin , may you guys take a look, please. Thank you!

@vladlearns
Copy link
Author

@Juanma-t, this should help you

@vladlearns vladlearns marked this pull request as draft January 28, 2026 10:09
@vladlearns
Copy link
Author

@leng-yue , @PoTaTo-Mika , @Stardust-minus , @Whale-Dolphin, I'm converting it to draft, for it not to be automatically closed. I've tested everything out, everything works. Please, take a look, it still needs you attention. Thank you!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant