Lead Data Engineer

Overview

In this role you will deliver actionable insights to improve business performance for Electricity Transmission. You’ll partner with stakeholders to translate challenges into data requirements and manage insight delivery across the full software lifecycle. The role blends data analytics with hands-on data product development on cloud platforms, notably Azure. You will contribute to creating trusted data assets and automated data pipelines that drive action and continuous improvement.

Responsibilities

• Understand data fabric, lake and warehousing principles and contribute across major platforms (e.g., Snowflake, Azure).
• Develop ETL for the insights platform and partner with ET teams to ensure data quality and impactful data models.
• Lead installations and patches for the insights platform and support upstream/downstream stakeholders.
• Support data flow and mapping for IT project designs.
• Produce documentation from requirements to delivery.
• Improve data quality to establish a single source of truth used by the business.
• Design, implement and maintain CI/CD pipelines and test frameworks for data pipelines and dashboards (Azure DevOps, GitHub Actions, etc.).
• Promote good software lifecycle practices including version control, code reviews, automated testing, and release management.
• Implement monitoring, logging and observability for data pipelines and dashboards; act on alerts.
• Collaborate with ML/agentic automation teams to integrate agent-based workflows where applicable.
• Apply ORM and database design principles in data models and mappings.
• Ensure data quality and contracts are embedded in pipelines with remediation of upstream issues.
• Continue data fabric/lake/warehouse design, ETL/ELT development, stakeholder engagement and single-source-of-truth usage.
• Provide day-to-day operational support for AI agents and automation capabilities.
• Monitor AI agent performance, availability, accuracy, and user adoption; troubleshoot issues.
• Identify AI-driven process improvements with stakeholders.
• Maintain AI agent knowledge bases, prompts and configurations; ensure governance, security and privacy compliance.
• Support testing, validation and deployment of AI agent features; analyze feedback to drive improvement.
• Ensure AI agents deliver reliable, accurate, and business-aligned outcomes; report on performance and value.
• Drive continuous improvement to enhance AI effectiveness and user experience.
• Support safe and responsible deployment of AI capabilities across the organisation.

Key requirements

• Demonstrable experience delivering business benefits through data products and dashboards from requirements to deployment and support.
• Proven ability to design and operate CI/CD for data platforms and analytics apps.
• Familiarity with SDLC practices: version control, code review, unit/integration testing, release management.
• Experience or understanding of agentic LLM or automation frameworks and safe production deployment of automated agents.
• Knowledge of ORM concepts in data models and service layers.
• Strong analytics and dashboarding experience (Power BI) and UX optimization for decision making.
• Hands-on experience building data products on Snowflake and/or Azure (Synapse, ADLS, Azure SQL, Data Factory).
• Experience working with stakeholders to define KPIs, SLAs and data contracts.
• Desirable familiarity with DBT, DataOps and modern transformation tooling.
• Experience supporting digital products, automation platforms, AI solutions, or enterprise apps.
• Understanding of Generative AI, LLMs, AI agents and conversational AI concepts.
• Strong analytical and problem-solving skills for diagnosing issues.
• Ability to translate business requirements into technical solutions.
• Knowledge of data governance, info security and responsible AI principles.
• Ability to interpret performance metrics to drive improvements.
• Experience in Agile delivery environments.

• Collaborative
• Analytical
• Problem-solving
• Snowflake
• Azure (Synapse, ADLS, Azure SQL, Data Factory)
• Power BI

…