Overview
In this role, you design and evolve Wayve’s data pipeline and orchestration platform to handle next-generation autonomous driving data. You’ll revamp the Harness system and build a DAG-based workflow platform, enabling scalable, reliable, and accessible data processing across teams. You’ll partner with cross-functional teams to define pipelines that drive high-priority data runs, batch inference, and evaluations, while focusing on reliability and cost-effective performance. This is a hands-on, impact-oriented opportunity to shape a critical data infrastructure at Wayve.
Pay / Benefits
- salary benchmarked against market
- meaningful equity
- relocation support and visa sponsorship
- hybrid working with core hours
- learning and development budgets
- comprehensive health insurance and dental; enhanced maternity/paternity leave
Responsibilities
- Design, build and evolve scalable data pipeline and orchestration infrastructure for the next-generation autonomous driving data platform
- Revamp the existing Harness system and develop a new DAG-based workflow platform
- Build tooling and libraries to simplify ingestion, processing, inference and evaluation workflows across teams
- Ensure hundreds of parallel pipeline steps run with correct outputs, no conflicts and prioritized resources
- Support batch inference and performance optimisation for cost-effective, scalable evaluations
- Improve platform reliability, quality and usability, including incident response workflows aligned to business-defined SLOs
Key requirements
- Strong software engineering fundamentals
- Production-grade systems design and maintenance
- Solid Python programming skills
- Experience with large-scale distributed systems or high-throughput data processing
- Knowledge of software engineering best practices (unit/integration tests, TDD, data-driven development)
- Familiarity with distributed event streaming platforms (e.g., Apache Kafka)
- Understanding of database principles and distributed data systems (Postgres, ClickHouse)
- Experience collaborating with platform users and partner teams to deliver scalable solutions
- Cross-functional collaboration
- User-focused problem solving
- Communication with partner teams
- Python programming
- Distributed systems / data pipelines
- Apache Kafka
…
