Saylee Bharsakle
ReportI am an early-career software engineer in India working on the serving and routing layer of open large language model (LLM) inference. In the past few weeks I have contributed upstream to vllm-project/aibrix (Kubernetes-native LLM serving), llm-d-router (request routing), and BerriAI/litellm (OpenAI-compatible request transformation). My merged work includes routing /v1/audio/speech through the aibrix gateway, an upload size-limit fix, unicode filename handling, and test reliability fixes, five merged pull requests in total. Two open fixes are in review in llm-d-router: the SGLang model-name rewrite and the prefix-cache block-hash seed, with further parser and transformation fixes open in litellm. I do not own these projects; I am a contributor to them. This grant funds dedicated time to ship a defined set of upstream fixes and to document the work in public so other Indian developers can enter the LLM-systems layer of FOSS. Open-source LLM inference infrastructure is where India's presence is thinnest relative to its developer population. The barrier is not hardware I own but sustained time to read, test, and land changes in large upstream codebases. Every deliverable is a public commit, pull request, or writeup on GitHub, reviewable and reusable by the upstream communities. The work needs no GPU of my own: it is verified on CPU through the upstream projects' own unit and integration test suites, which is how my existing contributions were validated.
vLLM Production Stack (aibrix)
The vLLM Production Stack (aibrix) is the Kubernetes-native control plane for serving vLLM at scale. I contribute upstream to the gateway plugin. My merged work includes routing /v1/audio/speech through the gateway, an upload size-limit fix, unicode filename handling, and test reliability fixes, five merged pull requests in total. This grant supports continued upstream work here, including gateway and plugin fixes, routing correctness, and test coverage. I am a contributor to this project, not its owner.
llm-d-router
llm-d-router is the request routing and endpoint-picking layer for disaggregated LLM inference in the llm-d project. I have two open pull requests: #3375 fixing SGLang model-name rewriting and #3378 fixing the prefix-cache block-hash seed so distinct (model, cache_salt) pairs no longer collide. This grant supports landing both and a set of parser and prefix-cache fixes with tests. I am a contributor to this project, not its owner.
LiteLLM
LiteLLM is the OpenAI-compatible proxy used by many self-hosted inference stacks. I contribute upstream to request and response transformation between provider APIs. This grant supports continued fixes to request transformation behavior and their regression tests. I am a contributor to this project, not its owner.
Development workstation (laptop or CPU machine)
One machine to build and test the upstream inference projects locally. The contributions are verified on CPU through the upstream projects' own Go and Python test suites, so no GPU is required, but a current laptop or CPU workstation is needed to run builds, race-enabled tests, and local model-serving experiments. Estimated cost about INR 50,000.
Inference infrastructure upstream contributions (6 months)
A six-month part-time fellowship. Milestones: (1) land my open fixes for the SGLang model-name rewrite (#3375) and the prefix-cache hash seed bug (#3378), continuing to a total of 6-10 tested upstream fixes across aibrix, llm-d-router, and litellm; (2) a fortnightly public build log; (3) one community engagement per quarter; (4) report from one Indian FOSS event. Each milestone is a public pull request, commit, or writeup. Covers stipend for six months; the workstation is requested separately.