AI Infrastructure Engineer
Netpreme, Santa Clara, CA or Boston, MA
The role
About the role
Serving frontier LLMs efficiently at scale is a moving target: every new model and workload reshuffles the tradeoffs across parallelism, batching, KV caching, and scheduling. You’ll own Netpreme’s inference-serving stack end to end, from tuning vLLM, SGLang, and TensorRT-LLM to architecting distributed serving with NVIDIA Dynamo (prefill/decode disaggregation, KV-cache-aware routing). Working directly with our hardware and systems engineers, you’ll build the benchmarks and drive the optimizations that take new models from research to production.
