Software Engineer, Inference Runtime
LM Studio, New York City
The role
As a team, we work with high technical intensity and personal responsibility. We are looking for curious, self-motivated, creative, and technically excellent teammates to join us and build the future of human-AI interactions in software.
We are looking for an Inference Runtime Software Engineer to push forward LM Studio's inference stack on-device and in the cloud. You will integrate new inference engines and runtime capabilities, bring up new open-weight models and modalities, and optimize model execution for a wide range of CPU and GPU targets. You will also contribute improvements to the open-source projects we build on.
What you will do
- Maintain and push forward our inference stack on-device and in the cloud
- Bring up new model architectures and multimodal models
- Improve latency, throughput, memory use, and reliability across CPU, CUDA, Metal, Vulkan, and ROCm runtimes
- Build runtime capabilities for model loading, batching, scheduling, caching, and distributed execution