Data Engineer, Data Quality & Provenance

Overview

In this role you design and operate scalable data pipelines and datasets that fuel autonomous-driving development. You’ll build discoverable, versioned data products and collaborate with ML, autonomy, simulation, and safety teams to enable reliable model training and evaluation. The work sits at petabyte scale, focusing on data quality, provenance, and cost-efficient, reliable storage and compute. You’ll help shape data standards and provide practical, production-grade platform capabilities for a growing team.

Pay / Benefits

• Salaries benchmarked against the market annually
• Meaningful equity
• Relocation support and visa sponsorship where applicable
• Hybrid working with core hours
• Learning and development budgets
• Comprehensive health benefits and wellbeing support

Responsibilities

• Design and operate scalable batch and streaming pipelines for multimodal fleet and simulation data
• Build data models, catalogs, indexes and query capabilities for sensor, vehicle-state, map and event data
• Create versioned, reproducible datasets for training, evaluation, replay, scenario mining and safety analysis
• Develop workflows for ingestion, synchronization, transformation, curation, labeling and quality validation
• Partner with autonomy, ML, simulation and safety engineers to define schemas, APIs and data contracts
• Improve reliability, performance and cost efficiency of large-scale storage and compute workloads
• Petabyte-scale data products supporting autonomous-driving development across ML, autonomy, simulation and safety teams
• Trusted datasets with strong versioning, lineage and reproducibility guarantees
• Metadata, geospatial and temporal indexing enabling data discovery and scenario mining
• Systems combining real-world fleet logs with simulation and synthetic data for closed-loop development
• Data-quality standards, observability, access controls, retention policies and incident-response processes
• Cloud-based storage and distributed-compute infrastructure designed for reliability, scalability and cost efficiency

Key requirements

• Strong hands-on Python and SQL skills with solid production software-engineering fundamentals
• Experience designing and operating large-scale distributed data systems
• Production experience with distributed-processing and workflow-orchestration technologies (Spark, Flyte or Airflow)
• Experience building cloud-based data platforms using object storage and governance
• Ownership of data quality, lineage, observability, reproducibility and incident response for production data workflows
• Ability to translate ambiguous requirements into durable, reusable platform capabilities
• Comfort working in ambiguity and shaping standards and operating models for a growing data platform
• Experience with autonomous vehicles, ADAS, robotics, mapping, sensor-rich systems or multimodal datasets is advantageous

• Collaborative and cross-functional work style
• Ability to operate in ambiguity and define standards
• Strong communication of complex data concepts to technical and non-technical stakeholders
• Python
• SQL
• Spark

…