Consultant/Senior Consultant, Databricks Data & AI Engineer, Engineering, AI & Data, Technology & Transformation

Overview

In this role you will design, build and optimise scalable data & AI solutions on the Databricks Platform to enable advanced analytics and AI-driven business outcomes. You will work with cross‑functional teams to modernise data platforms, implement robust data models, and embed governance and security. You’ll translate complex requirements into scalable, enterprise-grade data products that deliver measurable value. Join Deloitte’s AI & Data practice to help clients unlock trusted data foundations at scale.

Pay / Benefits

• hybrid working in London
• development opportunities
• supportive culture and wellbeing
• flexible working arrangements

Responsibilities

• Design, build, test and deploy scalable batch and streaming data pipelines and AI-ready data products on Databricks using Delta Lake, Spark SQL/PySpark, Unity Catalog and Databricks SQL
• Deliver the full data & AI engineering lifecycle from requirements through deployment and continuous improvement
• Engage with clients and stakeholders to gather requirements, run design workshops and propose practical solutions
• Communicate technical concepts clearly to both technical and non-technical audiences
• Design robust data models and ETL/ELT patterns supporting analytics, ML and AI use cases
• Optimize pipelines for performance, reliability, scalability and cost; identify risks and mitigations
• Incorporate security, governance, lineage, access control and data quality through Unity Catalog
• Enable AI/ML workflows by preparing governed datasets and supporting feature engineering, model deployment and integration
• Apply CI/CD, IaC, automated testing, monitoring and data observability practices
• Produce deployment documentation, runbooks and artefacts; contribute to engineering standards and QA
• Support and guide other engineers; participate in technical reviews and reusable standards
• Enable Gen AI & Agentic AI solutions with trusted, production-ready data foundations

Key requirements

• Data engineering experience, preferably on a major cloud platform (Azure, AWS, or GCP)
• Strong SQL and Spark (PySpark/Scala) skills with performance focus
• Experience with Databricks Lakehouse Platform and Delta Lake principles
• Hands-on delivery of enterprise-scale Databricks implementations
• Proficiency in Python or Scala for data engineering
• Experience gathering requirements from business and technical stakeholders
• Analytical, problem-solving and troubleshooting skills

• Collaborative mindset
• Excellent communication to varied audiences
• Proactive and initiative-taking
• Databricks Lakehouse Platform
• Delta Lake
• Unity Catalog

…