Lead Data Migration Engineer

Overview

In this role you will lead the design, build, and validation of end-to-end data migration pipelines from legacy warehouses to AWS-based Lakehouse platforms. You will collaborate with architects, engineers, and analysts to deliver secure, scalable migration solutions that ensure data quality and performance. You’ll work hands-on with AWS Glue, Iceberg, Python/PySpark, SQL, and YAML-driven configurations to modernise data infrastructure. This is a client-facing role focused on delivering migration excellence and reusable automation components. You will help shape data migration initiatives that underpin transformation across the data lifecycle.

Pay / Benefits

• flexible work options
• tailored benefits
• Learning and Development opportunities
• inclusive work environment
• commitment to diversity and inclusion
• Disability Confident Committed Employer

Responsibilities

• Serve as a senior engineer in data migration programmes, supporting delivery across complex transformations
• Collaborate with architects and stakeholders to implement migration strategies and technical solutions
• Plan, estimate, and execute migration deliverables
• Design, build, and optimise data migration pipelines from legacy warehouses to AWS-based Data Lakehouse platforms
• Develop and maintain ETL/ELT pipelines using AWS Glue, Python/PySpark, SQL, and YAML-driven configurations
• Implement bulk migration followed by incremental/delta loads
• Support pipeline repointing and replatforming to target cloud architectures
• Test end-to-end ETL data solutions on AWS services (AWS Glue, Apache Iceberg) and validate migrations to new Lakehouse architectures
• Develop SQL-based validation queries to ensure data completeness, accuracy, and transformation correctness
• Create automated testing and validation frameworks
• Collaborate with Solution Architects, Data Engineers, Analysts, and QA teams to promote best practices

Key requirements

• Proven experience in data engineering and data migration delivery in cloud environments
• Strong focus on data pipeline testing, validation, and quality assurance
• Experience across the full data lifecycle with emphasis on migration and transformation
• Strong analytical, problem-solving, and communication skills
• Client-facing and delivery-focused background
• Ability to mentor junior engineers and contribute to team delivery
• Hands-on experience with AWS, especially AWS Glue
• Python / PySpark for data transformation
• SQL for querying, validation, and reconciliation
• YAML configuration for pipeline orchestration
• Familiarity with Apache Iceberg and Lakehouse architectures
• Experience with distributed processing frameworks (e.g., Apache Spark)
• Understanding of ETL vs ELT design patterns

• collaboration
• client-facing communication
• problem-solving
• AWS Glue
• Python
• PySpark

…