Senior Data Engineer (Python / ETL / Entity Resolution)
- Senior, hands-on data engineering role with genuine ownership of production pipelines
- Python, SQL, AWS and Docker, with exposure to entity resolution, ML and NLP
- Flexible depending on experience
- Solve hard, real-world data problems for banks, governments and regulated organisations
About the Company
Our client builds identity, entity resolution and risk intelligence platforms used by banks, governments and regulated organisations to understand exactly who they are doing business with. Their software combines large-scale data processing, entity resolution, graph intelligence and modern cloud architecture. They are a small, high-trust engineering team with very little bureaucracy, where data quality sits at the heart of everything the product does.
The Role
You will join as a senior engineer working directly alongside the Principal Data Engineer, owning the pipelines that power an entity resolution platform processing millions of records. This is not a role for moving data between databases, you will be resolving identities and relationships across messy, overlapping datasets where correctness and performance genuinely matter.
You will take loosely defined problems from investigation through to production, with real influence over architecture and engineering standards. The client has a dedicated platform team, so you will deploy and operate your own workloads on AWS without carrying the heavy DevOps load. It suits someone who enjoys clean, pragmatic engineering and wants their work to make a visible difference to the product.
Key Responsibilities
- Own and evolve the Python ETL/ELT pipelines that ingest, normalise and enrich large, complex datasets
- Build and optimise data preparation workflows for entity resolution, from source mapping and record preparation to ingestion and downstream extraction
- Integrate structured, semi-structured and unstructured data from external providers, APIs, databases and documents
- Design robust data quality and validation processes covering completeness, duplication, provenance and integrity
- Build reliable batch and asynchronous processing with sound scheduling, retries, idempotency and failure recovery
- Containerise and deploy Python data workloads using Docker and AWS
- Develop data preparation pipelines supporting machine learning, OCR and NLP use cases
- Implement observability across your pipelines, including structured logging, metrics and alerting
- Contribute to architecture decisions, code reviews and technical documentation
What You’ll Need
Essential
- 3+ years of professional data engineering or software engineering experience, with hands-on responsibility for production data systems
- Strong Python, writing well-structured, maintainable, production-quality code
- Proven experience building ETL/ELT pipelines for complex or high-volume datasets
- Strong SQL and experience with relational databases such as PostgreSQL
- Solid grounding in data modelling, schema design, normalisation and validation
- Practical experience with Docker and AWS services such as S3, SQS, EC2 or Lambda
- Experience with automated testing, CI/CD and version control
- Based in the UK with the right to work in the UK
Nice to have
- Entity resolution, record linkage, deduplication or fuzzy matching experience
- Data science or ML exposure, such as scikit-learn, PyTorch, spaCy or OCR tooling
- Kubernetes (ideally EKS) or a background in KYC/AML, fraud or other regulated data environments
Why Apply ?
- Excellent salary, depending on experience
- Work on genuinely difficult technical problems rather than routine data plumbing
- High ownership and real influence over the systems you build, in a team with minimal bureaucracy
- Work directly alongside a Principal Data Engineer, with strong opportunities to grow your technical depth
- Modern stack spanning Python, AWS, Kubernetes, entity resolution, ML and LLM-based extraction
Interested? Connect with Ryan Quinn on LinkedIn ( or send your CV via the link below.
Skills:ETL SQL AWS ML
Benefits:Bonus
WHJS1_NI
…