Senior Data Engineer

Overview

In this senior role, you will architect and lead the migration of a large-scale ETL estate from on-prem tools to AWS. You will liaise with client engineers to reverse-engineer the current landscape and define the target state architecture across data processing, workflows, and operating models. You’ll design reusable migration patterns, validate inventories, and run a proof-of-concept to de-risk the approach. This engagement emphasizes data engineering excellence, architectural leadership, and delivery planning within a Tier 1 financial services environment.

Responsibilities

• Design, build, and maintain ETL/ELT pipelines on AWS
• Migrate existing data pipelines and workloads from legacy/on-prem systems to AWS
• Develop and optimize data models for data warehousing (Redshift, data lake on S3)
• Build and orchestrate data workflows (Glue, Step Functions, Lambda, EMR)
• Implement data quality checks, validation, and reconciliation processes
• Optimize pipeline performance and manage compute/storage costs
• Ensure data security and access control (IAM, KMS, encryption)
• Monitor and troubleshoot pipeline failures (CloudWatch, logging)
• Collaborate with data analysts, BI developers, and architects on data requirements
• Document data pipelines, architecture, and operational runbooks

Key requirements

• 5+ years of experience
• Hands-on experience building data pipelines on AWS (Glue, EMR, Lambda, Step Functions)
• Strong SQL and experience with data warehousing (Redshift, dimensional modeling)
• Proficiency in Python or Scala for data engineering
• Experience with S3-based data lake architecture (partitioning, cataloging, formats like Parquet)
• Experience migrating data pipelines from on-prem or other cloud platforms
• Understanding of data governance, security, and access control on AWS
• Experience with CI/CD for data pipelines

• collaboration with cross-functional teams
• ability to influence stakeholders
• structured delivery planning
• AWS data engineering stack (Glue, EMR, Lambda, Step Functions)
• SQL and data warehousing (Redshift)
• Python or Scala for data engineering

…