Lead Data Engineer

Job Description

n

Lead Data Engineer | 6 Month Contract | (Inside IR35) | Hybrid, 2 days pw Edinburgh/Glasgow | Starting ASAP

n

n

Day Rate: DOE

n

n

About the role:

n

Harvey Nash's Pub Sec client is seeking an experienced AWS Data Engineer to design, develop, and support scalable data warehouse and cloud-based data solutions. The successful candidate will work within an Agile Scrum team, delivering end-to-end data engineering solutions, ETL pipelines, and analytics platforms that support business intelligence and digital applications.

n

n

You will:

n

n
• Design, build, and maintain cloud-based data warehouse and data lake solutions on AWS.

n

• Develop and support ETL/ELT processes for data ingestion, transformation, and integration.

n

• Build and optimise data pipelines using Python, SQL, Spark, and AWS services.

n

• Develop serverless and event-driven solutions using AWS Lambda, Step Functions, EventBridge, SQS, and SNS.

n

• Support and enhance existing data platforms, applications, and integrations.

n

• Work closely with business and technical stakeholders to deliver solutions in 2-week Agile sprints.

n

• Create and maintain dashboards, reporting, and monitoring solutions using QuickSight, Power BI, Grafana, and ELK.

n

• Implement data quality, security, governance, and operational monitoring best practices.

n

n

n

Essential Criteria

n

Strong experience with AWS data and analytics services including:

n

n
• Redshift

n

• Glue

n

• S3

n

• Athena

n

• Lambda

n

• Step Functions

n

• RDS/Aurora

n

• DynamoDB

n

• DMS

n

• EMR

n

• CloudWatch

n

• IAM

n

n

Strong programming experience in Python and SQL.

n

Experience with Spark (PySpark preferred) and large-scale data processing.

n

Experience developing ETL/ELT pipelines and data warehouse solutions.

n

Knowledge of API integration and microservices architectures.

n

Experience working in Agile/Scrum delivery environments.

n

n

n

This role has been deemed Inside IR35 by the client. Applicants must hold, or be happy to apply for, a valid Basic Disclosure Scotland. Please click the link to apply.

n

…