Senior Data Engineer

Overview

In this role you will design and scale Clarion’s modern data platform, enabling analytics, operational data flows and future AI capabilities. You will work with cross-functional teams to implement robust data pipelines and a Medallion Architecture-driven data model. The role focuses on engineering excellence, governance, and production-ready Databricks solutions. This is a chance to shape scalable data foundations for global events and digital products.

Pay / Benefits

• 25 days holiday plus bank holidays
• End of year wellbeing shutdown
• Personalised extra leave day
• Summer Hours on Fridays
• HOW Days – paid charity day
• Pension Scheme

Responsibilities

• Design, build, and optimise scalable data pipelines using Databricks, Delta Live Tables, Synapse, and Delta Lake
• Develop ingestion, transformation, and orchestration workflows across APIs, SQL, SFTP, SaaS platforms
• Implement data quality, lineage, and governance aligned to Medallion Architecture (Bronze/Silver/Gold)
• Build and optimise Lakehouse data models for analytics, reporting, operational workloads, and downstream product teams
• Develop reusable frameworks for ingestion, validation, schema evolution, and monitoring
• Ensure Databricks best practices around Unity Catalog, cluster policies, permissions, and environment separation
• Collaborate with architects to evolve platform patterns, security, and performance
• Implement monitoring, logging, and observability across pipelines and clusters
• Contribute to standards for versioning, DevOps, CI/CD, and testing for notebooks, workflows, and SQL assets
• Support rollout of Delta Sharing, Unity Catalog governance, and Iceberg/Delta interoperability
• Partner with analysts and data scientists to ensure high-quality, well-modelled data for analytics and AI use cases
• Assist with productionising ML pipelines within Databricks and support feature engineering and experiment environments
• Work cross-functionally with product, marketing, operations, and technology teams to understand data needs and accelerate delivery
• Mentor junior engineers and foster engineering best practices
• Champion DevOps for data: CI/CD pipelines, testing automation, IaC, and environment management

Key requirements

• 5+ years in data engineering in cloud-native environments
• Expert knowledge of Databricks, Delta Lake, Unity Catalog, ADF, Synapse, Event Hub, and Azure Data Services
• Proficiency in Python and SQL with production-grade data pipelines
• Strong understanding of Lakehouse design, Medallion modelling, and performance optimisation
• Hands-on experience with CI/CD for data (Azure DevOps, YAML pipelines, Terraform, Databricks CLI/Repos)
• Solid grasp of data governance, security principles, and operational best practices

• Mentoring and coaching
• Cross-functional collaboration
• Engineering best-practice mindset
• Databricks
• Delta Lake
• Unity Catalog

…