Member of Technical Staff - ML Systems & Inference

• Strong software engineering fundamentals
• Experience building or operating ML inference or model serving systems
• Comfort reasoning about performance, memory usage, and system behavior under load
• Bachelor's degree in a relevant field, or an equivalent combination of education, training, and professional experience.

• Improve the latency, throughput, and efficiency of production inference workloads
• Design execution strategies across batching, scheduling, concurrency, and resource utilization
• Improve KV cache management, memory efficiency, and execution under load
• Enable new models, accelerator architectures, and inference techniques to run efficiently in production

• Experience with inference runtimes such as TensorRT-LLM, vLLM, or custom serving systems
• Deep understanding of modern model architectures and attention mechanisms
• Experience with batching, scheduling, and concurrency control in inference systems
• Familiarity with KV cache management and memory placement strategies
• Experience profiling and tuning latency- and throughput-critical systems
• Software development experience in Python and C++

Why join now?

Gimlet is expanding from its core technology into a production neocloud spanning new hardware, customers, and data centers.

• Solve hard problems.
• Own meaningful work.
• Build for production.
• Help define what’s next.

Agency Policy: Gimlet Labs does not accept unsolicited resumes from recruitment agencies or search firms. Any unsolicited resumes submitted without a signed agreement will be considered the property of Gimlet Labs, and no fees will be paid.

Gimlet is building the first multi-silicon neocloud designed for fast, efficient AI inference.

We combine large-scale compute infrastructure with an execution platform that partitions AI workloads and maps each stage to the hardware best suited to run it.

We work with foundation labs, hyperscalers, and AI-native companies, giving our team access to technical problems spanning frontier models, production infrastructure, and emerging hardware.