Principal Data Engineer, Data Quality & Datasets Pipelines

Overview

In this role you will shape how Wayve moves vast driving data from ingestion to intelligent filtering. You’ll build data pipelines and retention policies that scale to hundreds of petabytes, balancing compute and data value. You’ll work closely with the dataset strategy lead and data infra teams to deliver practical, fast-moving solutions. This is a data-centric engineering role focused on quality, efficiency and real-world impact on model training. Join a mission to unlock scalable autonomy through data-driven decisions.

Pay / Benefits

• hybrid working model
• equity and long-term ownership
• relocation support and visa sponsorship
• learning and development budgets
• health insurance and dental
• pension or retirement benefits

Responsibilities

• Implement filtering, retention, and offload policies across hot, cold, archive and delete storage
• Build data movement and pipelines for pre-ingestion selection models
• Decide where and how selection models run to balance compute cost and data value
• Create early, low-compute decision points to avoid expensive data standardisation
• Collaborate with storage and data infra teams and monitor data quality to reflect model training needs
• Turn dataset strategy into working systems handling hundreds of petabytes
• Engineer the pre-ingestion selection model flow: data movement, model run location, offload policies
• Deliver large storage-cost savings via early, low-compute decisions
• Ensure datasets contain useful behavioural and environmental coverage for training
• Work closely with dataset strategy lead and data infra to ship practical solutions

Key requirements

• senior data or software engineer with petabyte-scale data ingestion or data quality pipelines
• strong Python and Spark skills with cloud data engineering experience
• good judgment on data filtering and quality; avoid discarding valuable data
• ability to ship practical solutions quickly and iterate on live systems
• software engineer at heart, building systems around models rather than models themselves
• experience collaborating with data infra and storage teams
• bonus: AV, robotics, fleet data or high-volume IoT experience or knowledge of retention/offload data lifecycle policies

• quick learner
• practical problem solving
• cross-functional collaboration
• Python
• Spark
• cloud data engineering

…