Data Ingestion Engineer

We’re looking for a Data Ingestion Engineer with experience working in the robotics industry. You will work with the Data team to improve the reliability, efficiency and throughput of the pipelines that move real-world driving data through the client.

Key Responsibilities:

• Debug and fix failing or blocked ingestion pipelines.
• Investigate issues caused by corrupt, malformed or unexpected data.
• Help make our pipelines more resilient so individual bad data segments do not block retries and queue management.
• Improve how we handle varied data formats from partners, suppliers and third-party sources.
• Optimise Spark jobs and data processing pipelines for throughput and compute
• Reduce operational toil around failed jobs, stalled pipelines and manual interventions.
• Work on high-volume batch processing systems where throughput, reliability and cost all matter.
• Help prioritise and unblock important datasets for downstream annotation, data science and model training teams.
• Partner with permanent engineers to keep critical ingestion work moving while longer-term platform improvements are developed.
• Contribute pragmatic improvements that make the system easier to operate, scale and trust.
• Strong production experience with Apache Spark.
• Experience building, debugging or operating large-scale data ingestion, ETL or data
• Experience with distributed data processing systems.
• Ability to optimise jobs for throughput, compute efficiency and reliability.
• Experience working in the robotics sector

#J-18808-Ljbffr…