Jobs Ask AI Pay Companies CompareSign in

Wait…

We read every job from the company's own job board. You apply on the company site.Jobs Job types Internships New graduate jobs Companies Pay Reported pay Check an offer Add your pay Compare with LinkedIn Job sites compared Pay transparency laws API for developers Monthly report Companies we cover Make a portfolio page For companies: find people About Privacy

AI Inference Platform Engineer

DRW , Chicago

Posted pay

$200K–$250K

a year

Location: Chicago
Work type: Not given
Level: Not given
Skills in the job text: CUDA
Grafana
Linux
Prometheus
vLLM
Posted: Sep 29, 2026
Last check: 5 h ago
Job type: Inference & ML systems, ML / AI engineering

Apply on the DRW site Sign in to save

The DRW site opens in a new tab.

D

DRW

About DRW : funding, hiring and pay

Check my fit

Compare your résumé with this job. See where you match, what is missing and which lines to change.

Sign in to check your fit

The role

We're looking for an AI Inference Platform Engineer to build, operate, and optimize the systems that serve large language, vision, multimodal, and embedding models across DRW. This role provides DRW's firmwide interface to modern AI models, from early evaluation through reliable production use.

You'll work across inference runtimes, distributed systems, and production platform engineering, with deep GPU literacy. You'll own the serving platform end-to-end: onboarding newly released models, measuring quality and performance equivalence across serving configurations, scheduling workloads across tenants, and continuously improving latency, throughput, utilization, reliability, and cost across the inference fleet.

What you will do

Optimize LLM inference performance across modern NVIDIA GPU architectures and inference runtimes.
Build end-to-end performance profiling and observability to identify bottlenecks from individual GPU kernels through multi-node inference systems.
Design and optimize KV cache and distributed inference architectures, including caching, routing, memory tiering, and prefill/decode strategies.
Own day-0 model onboarding, determining the appropriate runtime, precision, sharding, memory, batching, cache policy, and serving configuration for new models.
Maintain validated performance profiles for important model and hardware combinations, including performance and quality regression testing.
Measure and monitor quality equivalence across serving configurations, including KV cache quantization, speculative decoding acceptance thresholds, precision choices, and model routing, so in-house serving can be trusted to match reference-model quality on production workloads.
Manage the production serving lifecycle of models, including versioning, compatibility, staging, canarying, promotion, rollback, and retirement.
Partner with SRE and platform teams to automate model deployment, distribution, production readiness, observability, and reliable operation across environments.
Optimize model placement, scaling, and resource allocation across the inference fleet to improve utilization and cost efficiency while meeting performance and reliability requirements.
Design and operate multi-tenant scheduling and isolation across shared GPU capacity, balancing latency SLOs, throughput, and priority across concurrent workloads.

The Tech

Hands-on experience serving LLMs on NVIDIA GPUs, with familiarity across current and emerging architectures (Hopper, Blackwell, and successors), HBM, Tensor Cores, NVLink/NVSwitch, and the compute and memory bottlenecks that shape serving decisions.
Deep expertise in at least one modern inference runtime such as TensorRT-LLM, vLLM, or SGLang.
Practical knowledge of inference optimization techniques including continuous batching, scheduling, chunked prefill, speculative decoding, quantization, CUDA Graphs, and paged attention.
Understanding of KV cache architecture, including prefix caching, block management, sizing, eviction, quantization, cache-aware routing, and multi-tier caching.
Experience measuring model quality equivalence across serving configurations, including evaluation harnesses, task-specific benchmarks, and regression detection for quantization, KV cache, and speculative decoding changes.
Experience designing and tuning distributed inference systems, including tensor parallelism, multi-node deployments, and disaggregated prefill and decode.
Experience with multi-tenant GPU scheduling, workload isolation, and QoS across concurrent inference workloads.
Proficiency with GPU performance and observability tooling such as Nsight, DCGM, OpenTelemetry, Prometheus, and Grafana.
Strong Linux and systems performance fundamentals, with the ability to diagnose bottlenecks across hardware, drivers, runtimes, networking, and application layers.
Production experience with model serving infrastructure, including CI/CD, automated testing, observability, and production readiness.

The Intangibles

You take a measurement-driven approach to performance optimization.
You take ownership of performance problems across hardware, runtime, model, and infrastructure boundaries.
You can move quickly and reprioritize as trading needs change, while maintaining a high bar for production systems.
You understand the importance of reliability, predictability, and performance when AI systems are integrated into trading workflows and decision-making processes.
You can evaluate unfamiliar models, runtimes, and hardware quickly and make sound engineering decisions with limited prior guidance.
You communicate clearly and can explain complex performance tradeoffs across engineering teams.

The annual base salary range for this position is $200,000 to $250,000 depending on the candidate’s experience, qualifications, and relevant skill set. The position is also eligible for an annual discretionary bonus. In addition, DRW offers a comprehensive suite of employee benefits including group medical, pharmacy, dental and vision insurance, 401k (with discretionary employer match), short and long-term disability, life and AD&D insurance, health savings accounts, and flexible spending accounts.

For more information about DRW's processing activities and our use of job applicants' data, please view our Privacy Notice at https://drw.com/privacy-notice.

California residents, please review the California Privacy Notice for information about certain legal rights at https://drw.com/california-privacy-notice.

About DRW

DRW is a diversified trading firm with over 3 decades of experience bringing sophisticated technology and exceptional people together to operate in markets around the world. We value autonomy and the ability to quickly pivot to capture opportunities, so we operate using our own capital and trading at our own risk.

Headquartered in Chicago with offices throughout the U.S., Canada, Europe, and Asia, we trade a variety of asset classes including Fixed Income, ETFs, Equities, FX, Commodities and Energy across all major global markets. We have also leveraged our expertise and technology to expand into three non-traditional strategies: real estate, venture capital and cryptoassets.

We operate with respect, curiosity and open minds. The people who thrive here share our belief that it’s not just what we do that matters–it's how we do it. DRW is a place of high expectations, integrity, innovation and a willingness to challenge consensus.

Read the job posting on the DRW site

Similar jobs at other companies

DT
Inference Engineer - Member of Technical Staff
Designworks TalentBellevue
- Hybrid
- Inference & ML systems
- Member of Technical Staff
Pay not posted
Posted today
E
Inference Infrastructure Engineer, Serving
ElorianPalo Alto
- Hybrid
- Inference & ML systems
- Visa sponsorship in the job text
$200K–$400K / yr
Posted Jul 22, 2026
Senior AI Research Engineer - LLM Training & Inference Infrastructure
DatadogParis, France +1
- Senior
- Research
Pay not posted
Posted yesterday
Senior Software Engineer I, Inference
CoreWeaveSunnyvale, CA / Bellevue, WA +2
- Senior
- Inference & ML systems
$139K–$204K / yr
Posted Jan 23, 2026
Senior Software Engineer, Inference
CoreWeaveSunnyvale, CA / Bellevue, WA +2
- Senior
- Inference & ML systems
$152K–$204K / yr
Posted Sep 26, 2025
Senior Software Engineer - AI Inference Performance
NVIDIAUS, CA, Santa Clara
- Senior
- Inference & ML systems
$184K–$357K / yr
Posted 12 days ago

Source: the DRW Greenhouse job board. We last checked this job 5 h ago. We first saw it 4 days ago. A job on a job board does not always mean that the team interviews people now. We found this job board in a public list. We did not yet compare it with the careers page of the company.

DRW

Open tech jobs: 107
With posted pay: 53%
Median posted base: $175K–$225K

About DRW : funding, hiring and pay

Compared with similar jobs

This base pay is above the median for inference and ML systems jobs in other US places with no level in the job title.

Median: $197K a year
Other jobs in the group: 105

We compare the middle of each posted range. This is base pay as posted. The group does not include this job.