Distributed LLM Inference Engineer
Anyscale, San Francisco +1
The role
As a Distributed LLM Inference Engineer, you will help systems and optimizations that push the boundaries of performance for inference at large scale. This is an incredibly critical role to Anyscale as it allows us to achieve a market leading position for AI infrastructure.
What you need
- Familiarity with running ML inference at large scale with high throughput and low latency
- Familiarity with deep learning and deep learning frameworks (e.g. PyTorch)
- Solid understanding of distributed systems, ML inference challenges