Senior Data Engineer (AI/Tooling)

Overview

In this strategic data engineering engagement, you will architect and plan a large-scale migration of ETL estates from Matillion to AWS Glue for a leading financial services client. You’ll reverse-engineer current pipelines, design target architecture across data, processing, and governance, and deliver a blueprint for multi-phase migration. The role blends hands-on engineering with architecture leadership and cross-functional collaboration. You’ll shape the migration approach and work closely with governance boards to enable a multi-million-pound program.

Responsibilities

• Agentic AI Migration Framework development to automate and speed up migration workflows
• Pattern library build: reusable migration patterns and templates for common conversions
• Automated conversion pipeline: transforming Matillion jobs and configurations during migration
• GitHub Copilot integration to accelerate code generation and conversions
• Support factory pods: provide tooling, support, and troubleshooting for scalable migrations

Key requirements

• Python — strong hands-on development for building tools and automation
• GenAI/LLM integration — experience embedding GenAI capabilities into apps/workflows
• Prompt engineering — design and optimise prompts for LLM tools
• AWS Bedrock — hands-on experience for GenAI-powered apps
• GitHub Copilot — experience integrating into development workflows
• Automation frameworks — building end-to-end automation tooling
• Testing — test automation and validation for conversion accuracy
• Matillion JSON parsing — parsing and transforming Matillion job definitions (JSON)

• collaboration with cross-functional teams
• stakeholder influence
• structured delivery planning
• Python
• GenAI/LLM integration
• Prompt engineering

…