- BS, MS, or PhD in Computer Science, Computer Engineering, Electrical Engineering, Applied Math, or a related field, or equivalent experience.
- 3+ years of relevant engineering experience.
- Must have: Extensive knowledge of the efficiency and optimization involved in AI model execution, covering continuous batching, throughput-latency tradeoffs, KV cache and memory limitations, parallel processing techniques, MoE serving, quantization, and meeting serving SLAs.
- Must have: Hands-on experience benchmarking and profiling GPU workloads using tools such as Nsight Systems, Nsight Compute, CUPTI, or PyTorch profiler, and interpreting kernel-level performance data.
- Strong Python engineering skills and the ability to navigate and modify large C++/CUDA serving codebases.
- Rigorous experimental methodology with controlled single-variable comparisons, reproducible benchmarks, and evidence-backed optimization decisions.
- Strong written and verbal communication skills to explain performance tradeoffs clearly to both humans and documentation for autonomous systems.
Ways to stand out from the crowd
- Direct contributions to TensorRT-LLM, vLLM, SGLang, FlashInfer, Dynamo, or comparable inference frameworks.
- Experience with disaggregated serving, wide expert-parallel MoE inference, KV cache transfer, or NCCL/NIXL/NVSHMEM communication at multi-node scale.
- CUDA kernel authorship or optimization experience on Hopper/Blackwell architectures, focusing on Tensor Cores, TMA, and warp specialization.
- Proven results on public inference benchmarks such as MLPerf Inference or SemiAnalysis InferenceX.
- Experience building or operating agentic AI workflows to automate engineering tasks.
Widely considered to be one of the technology world’s most desirable employers, NVIDIA offers highly competitive salaries and a comprehensive benefits package. As you plan your future, see what we can offer to you and your family www.nvidiabenefits.com/
Your base salary will be determined based on your location, experience, and other objective job-related factors. The expected base salary ranges for this position are: 124,000 USD – 195,500 USD for Level 2; and 152,000 USD – 241,500 USD for Level 3.
You will also be eligible for equity and benefits.
Applications for this job will be accepted at least until November 13, 2026.
This posting is for an existing vacancy.
NVIDIA uses AI tools in its recruiting processes.