Backend Engineer – API

Overview

In this role you will help build the xAI API that exposes models to developers worldwide. You will own the end-to-end system for high-throughput, low-latency inference with high availability. The position sits in a hands-on, flat-structure team focused on engineering excellence and initiative. You will work on scalable model-serving infrastructure, routing, SDKs, observability, and efficient scaling to support billions of tokens per minute. You join a mission-driven environment that values curiosity and clear, concise communication.

Responsibilities

• Build the xAI API that serves models to developers worldwide
• Own end-to-end system for high-throughput inference with low latency and high availability
• Work on model serving infrastructure, request routing, SDK development, rate limiting, observability, and scaling

Key requirements

• Expert knowledge of either Rust or C++
• Experience in designing, implementing, and maintaining reliable and horizontally scalable distributed systems
• Knowledge of service observability and reliability best practices
• Experience operating databases such as PostgreSQL, Clickhouse, and MongoDB

• Strong communication skills
• Hands-on mindset and initiative
• Good prioritization and time-management
• LLM inference engines and serving frameworks (e.g., SGLang, TensorRT, vLLM)
• Experience with agent SDKs and agent orchestration frameworks
• Docker and Kubernetes

…