JobsWithPay

Inference and ML systems engineer job description

An inference and ML systems engineer makes machine learning models run fast and at low cost. The engineer works on model serving, compilers, GPUs and the software between the model and the hardware.

Numbers from 12,029 open job postings at 2,715 companies. We read each posting from the job board of the company. Data of . The pay numbers are base pay ranges that employers posted in job postings. They are not salaries that people reported.

Open job postings
12,029
Companies
2,715
Median posted base pay
$219K
Experience in the middle posting
5 years
Postings that show pay
53%

Inference and ML systems engineer duties and responsibilities

These are the usual duties of the job. One job at one company can have more duties or different duties.

Skills that employers ask for

The skills in the most inference and ML systems job postings are Python (60%), C++ (28%), PyTorch (26%), AWS (22%) and Kubernetes (22%). A skill name opens the open jobs that ask for it.

The percent is the part of the 12,029 open postings whose text names the skill. We read named languages, tools and platforms. We do not read general skills such as communication. A skill shows here when 3% of the postings or more name it.

Experience, level and work location

The middle posting asks for 5 or more years of experience. The tables show what the open postings state.

Years of experience

Minimum in the textPostingsPart
0 to 2 years6398%
3 to 5 years3,59748%
6 to 9 years2,13728%
10 years or more1,14715%

From the 7,520 postings that state a minimum.

Level

Level in the job titlePostingsPart
Intern5875%
Entry level2922%
No level in the title5,57146%
Senior3,44129%
Staff1,0789%
Principal6636%
Director and above3973%

From all 12,029 postings.

Work location

Work typePostingsPart
On site2,58846%
Hybrid1,55828%
Remote1,49827%

From the 5,644 postings that state a work type.

Job titles that employers use

The job titles of this job type that the most companies use in open postings. One job type has many titles.

Inference and ML systems engineer salary: posted pay by level

The median posted base pay for inference and ML systems jobs is $219K a year, from 5,814 open jobs at 1,452 companies that state a yearly range in US dollars. The middle half of the jobs is from $183K to $265K. These are pay ranges that employers posted. They are not salaries that people reported, and they do not include stock or a bonus.

Pay bands by level, inference and ML systems. Posted base pay in US dollars for each year.
Level10th25thMedian75th90thJobs with pay
Intern$66K$79K$118K$154K$179K25
Entry level$90K$126K$151K$169K$216K78
Level not stated$144K$169K$205K$254K$313K2,651
Senior$158K$188K$215K$245K$271K1,765
Staff$193K$220K$249K$279K$326K686
Principal$181K$214K$256K$320K$364K384
Director+$189K$231K$276K$309K$353K225

The companies with the highest median posted base pay for these jobs: Netflix ($608K), Anthropic ($441K), Thinking Machines Lab ($413K), Magic ($413K), Aaru ($376K) and Datadog ($360K).

An outline of a job post that you can copy

Copy this outline and change it for your job. The text in brackets tells you what to write. The duties are the usual duties of the job. The skills, the years and the pay come from the open postings on this page.

Job title: Senior Software Engineer

About the job
[Two or three sentences: what the team builds, who uses it, and why this job is open.]

What you will do
- Build and operate the services that run models for users.
- Measure latency, throughput and cost, and find the slow parts.
- Make models smaller or faster with methods such as quantization and batching.
- Write performance-critical code in C++, Rust or CUDA.
- Plan GPU capacity and schedule the work on the hardware.
- Work with researchers so that new models can run in production.

What we look for
- [Number] or more years of experience. [The middle posting for this job asks for 5.]
- Skills: Python, C++, PyTorch, AWS, Kubernetes, Linux, GCP, Azure. [These are the skills in the most postings. Keep only the skills that the job needs.]
- [One or two more requirements that are true for your team.]

Pay
[Minimum] to [maximum] US dollars a year, and a short description of the bonus, the stock and the benefits. [The middle half of the posted base pay for this job is $183K to $265K a year.]

Work location
[On site, hybrid or remote. The city and the state.]

How to apply
[The link, and what happens after a person applies.]

Pay in the job post: the rule in your state

14 US states and the District of Columbia require a pay range in a job post: Colorado, California, Washington, New York, Hawaii, District of Columbia, Maryland, Illinois, Minnesota, New Jersey, Vermont, Massachusetts, Virginia, Maine and Connecticut. Many of these laws also cover a remote job that a person in the state can do. See the pay transparency law of each place, with the official source.

Do you hire for this job?

Put your job in front of people who look for inference and ML systems jobs with the pay shown.

Post this job free

Do you look for this job?

12,029 inference and ML systems jobs are open now. 53% of them show the pay.

See the open inference and ML systems jobs

Newest inference and ML systems jobs

Read real postings to see how employers describe this job. Each job is from the job board of the company.

Do you want to apply? See the inference and ML systems engineer résumé example.

How we made this page

The numbers come from open job postings. We read each posting from the job board of the company and check it every day. A program reads the skills, the years of experience and the level from the text and the title of each posting. One job can be in more than one job type.

A person wrote the short description of the job and the list of duties. They are general: one job at one company can be different. The pay numbers are posted base pay ranges in US dollars for each year. They do not include stock or a bonus, and they are not salaries that people reported.