- Monokernel: a persistent GPU program that keeps the entire token generation loop on the GPU, avoiding CPU handoffs.
- KCCL: our custom inter-GPU communication layer, designed to minimize latency.
- Delayed Tensor Parallelism (DTP): a model architecture that overlaps communication with computation and weight loading.
- Laneformer 2B: our code-generation model demonstrating this co-design approach.
Our engineers work directly with GPU hardware, using CUDA, PTX, HIP, and CDNA ISA to optimize execution.
Implementing and optimizing a new AI model for a specific GPU architecture can take months of manual engineering. Meanwhile, new models are constantly being released.
Our goal is to use AI agents to implement and optimize the models we choose on the GPUs we choose, in a fraction of the time required today.
These agents will build on the GPU optimizations and hardware insights our team has developed. They'll explore optimization strategies, generate and modify kernels, verify correctness, benchmark performance on real hardware, and iterate based on the results.
We want to make model porting and GPU optimization much faster and repeatable, allowing Kog to keep pace with the latest AI models rather than spending months adapting each one manually.
This is a major engineering direction we're now developing, and you'll help shape it.
Learn more at https://blog.kog.ai.