Data Engineer

Overview

As a Data Engineer, you will build and maintain robust data pipelines in AWS, with a focus on Delta Lakehouse on Databricks. You will collaborate with data scientists and analysts to meet data requirements and ensure data quality and performance. The role emphasizes staying current with AWS and Databricks developments to drive scalable data solutions. This is an opportunity to shape data-driven products and contribute to growth through practical, technology-forward work.

Pay / Benefits

• 22 days holiday rising to 25
• Staff discounts & Friends and Family discounts
• Cycle to work scheme
• Tech Scheme
• Breakfast and drinks provided
• Charity day per annum

Responsibilities

• Build and optimize data ingestion pipelines using AWS Lambda, Kinesis, and Glue
• Develop and optimize transformation pipelines in Delta Lake on Databricks
• Design and implement efficient data models
• Collaborate with data scientists, analysts, and stakeholders
• Ensure data reliability, integrity, and timeliness through quality checks
• Improve query performance and data retrieval efficiency
• Provide technology stewardship by evaluating new tools and approaches

Key requirements

• Proficiency in AWS services Lambda, Kinesis, and Glue
• Hands-on Databricks and Delta Lakehouse experience
• Programming skills in Python, Scala, and SQL
• Experience designing, building, and maintaining ETL processes
• Experience in logical and physical data modelling with large datasets
• Strong analytical and problem-solving abilities

• collaboration
• analytical thinking
• problem-solving
• AWS Lambda
• AWS Kinesis
• AWS Glue

…