Lead Data Migration Engineer

Overview

Lead data migration engineer within the Data & AI practice, focusing on designing and validating end-to-end ETL/ELT pipelines to migrate data from legacy warehouses to AWS-based Lakehouse platforms. You will ensure data quality, integrity, and performance while collaborating with architects, engineers, and analysts to deliver scalable migration solutions. This role offers hands-on work with AWS Glue, Iceberg, and PySpark, contributing to modernisation and client delivery in a collaborative environment.

Pay / Benefits

• flexible work options
• learning and development opportunities
• focus on wellbeing
• competitive benefits
• inclusive work environment
• disability confident employer

Responsibilities

• Design, build, and optimize data migration pipelines from legacy warehouses to AWS-based Data Lakehouse platforms
• Develop and maintain ETL/ELT pipelines using AWS Glue, Python/PySpark, SQL, and YAML-driven configurations
• Implement bulk data migrations followed by incremental/delta loads and support replatforming to target cloud architectures
• Test end-to-end ETL data solutions on AWS and validate migrations to new Lakehouse architectures
• Create automated testing and validation frameworks for data completeness, accuracy, and transformation correctness
• Collaborate with architects, engineers, analysts, and QA to promote reusable components and migration accelerators
• Ensure data integrity, quality, traceability, and secure handling in regulated environments
• Contribute to audit and validation processes for GDPR and public sector data requirements

Key requirements

• Proven experience in data engineering and data migration delivery in cloud environments
• Strong focus on data pipeline testing, validation, and quality assurance
• Experience with end-to-end data lifecycle migration and transformation
• Strong analytical, problem-solving, and communication skills
• Client-facing and delivery-focused mindset
• Ability to mentor junior engineers and contribute to team delivery
• Hands-on experience with AWS Glue, Python/PySpark, SQL, and YAML configurations
• Familiarity with Apache Iceberg and Lakehouse architectures
• Experience with distributed processing frameworks (e.g., Apache Spark)
• Understanding of ETL vs ELT design patterns
• Desirable: experience with CI/CD and version control tools

• Collaborative
• Client-facing
• Mentoring
• AWS Glue
• Python
• PySpark

…