Senior Lead Software Engineer, Big Data & Cloud Engineering — Risk Central (London)

Overview

Senior hands-on engineering leader responsible for building and operating scalable data and analytics services for markets risk. You will shape technical direction across Databricks and Redshift in a cloud-first environment, partnering with Front Office, Risk, Product Control, and Finance to deliver timely, high-quality analytics. You’ll drive end-to-end data pipelines, governance, and performance, while mentoring engineers and championing operational excellence. This role offers the opportunity to impact risk decisions at scale within JPMorgan Chase’s Risk Central platform.

Responsibilities

• Lead end-to-end delivery across ingestion, transformation, storage, and consumption layers
• Promote engineering best practices: design and code reviews, testing standards, CI/CD, and documentation
• Mentor engineers to elevate technical quality and ownership
• Design robust batch and streaming pipelines for high-volume workloads
• Implement data modeling, partitioning, and performance tuning for large datasets
• Build reusable data frameworks to accelerate onboarding of new datasets and analytics use cases
• Engineer Databricks and Redshift data products and workflows (ingestion, transformations, serving)
• Define approaches to data reconciliation, consistency, lineage, and controls across data warehouses
• Optimize queries and manage cost with best practices for workload placement
• Develop cloud-native AWS solutions (S3, EMR, Lambda, Kinesis, Glue, EventBridge, DynamoDB, Redshift, EKS) for reliability and scalability
• Own production stability through monitoring, incident management, root-cause analysis, and SLAs/SLOs

Key requirements

• Production-grade software delivery at scale
• Proficiency in Python and/or Java/Scala with CS fundamentals
• Hands-on experience with distributed data processing (Spark) and batch/streaming pipelines
• Experience with Databricks, Redshift, or equivalent data warehouse/lakehouse platforms
• SQL proficiency and data modeling understanding
• Cloud engineering basics: security, networking, IAM, encryption, observability
• Ability to troubleshoot production issues and drive operational improvements
• Strong communication and experience working with globally distributed teams

• Strong communication
• Mentoring and leadership
• Cross-team collaboration
• Python/Java/Scala
• Spark
• Databricks

…