Software Engineer III – Cloud Data Platform (AWS/Databricks, Terraform)

Overview

As a Software Engineer III on the Cloud Data Platform team, you design, provision, and operate a modern cloud data platform on AWS and Databricks. You collaborate with data engineers, security, and platform teams to deliver scalable, secure, production-ready infrastructure that enables reliable analytics. You will build and maintain IaC, Databricks capabilities, and platform automation to reduce manual effort and improve consistency. This role combines hands-on engineering with security, observability, and governance to support impactful data workloads across the firm. You will work in a fast-moving environment that values craftsmanship, collaboration, and continuous improvement.

Responsibilities

• Design, build, and maintain AWS cloud infrastructure with Terraform and IaC best practices (networking, security, compute, storage, managed services)
• Develop and manage Databricks platform components (workspaces, clusters, policies, jobs, libraries, integrations) for production data workloads
• Create automation for platform provisioning and operations using Python or Java to standardize environments
• Build CI/CD pipelines for infrastructure and platform deployments with Terraform plans, policy checks, and automated tests
• Apply security best practices (least-privilege, secrets management, encryption, network segmentation, auditability)
• Improve reliability via observability tooling (logging, metrics, tracing), alerts, incident response, and cost optimization
• Collaborate with stakeholders to translate requirements into scalable designs and document standards and runbooks
• Leverage enterprise AI coding assist tools to improve code quality and delivery, with peer review and secure coding validation
• Enhance SDLC tooling with AI-assisted development and automation capabilities to maximize automation value

Key requirements

• Formal training or certification in software engineering concepts with practical experience
• Hands-on AWS infrastructure provisioning with Terraform (modules, remote state, workspace management, change workflows)
• Solid experience with core AWS services (VPC, IAM, S3, EC2/ECS/EKS, RDS, KMS, CloudWatch, load balancing)
• Experience with Databricks (platform administration and/or production workloads)
• Proficiency in Python (preferred) or Java, with automation and platform integrations
• Experience with Git workflows and CI/CD for infrastructure and platform delivery
• Strong troubleshooting across cloud infrastructure, distributed systems, and data pipelines
• Experience using enterprise AI-assisted development tools and ability to evaluate AI outputs
• Understanding of responsible AI use, data sensitivity, secure handling, and resiliency and security expectations

• collaboration
• problem-solving
• strong communication
• Terraform
• AWS
• Databricks

…