Production AI Engineer – Vice President

Overview

As a Production Engineer at Citi, you design, build, and operate production-grade software within a highly regulated financial tech environment. You will architect autonomous AI workflows, integrate LLMs into tooling, and advance AI-native observability to prevent incidents. You contribute to a multi-year transformation, shaping how production systems leverage autonomous agents and self-healing infrastructure. You work cross-functionally to deliver scalable solutions, mentor teams, and communicate progress across the organisation.

Pay / Benefits

• 27 days annual leave
• discretionary annual bonus
• Private Medical Care & Life Insurance
• Employee Assistance Program
• Pension Plan
• Paid Parental Leave

Responsibilities

• Design, develop, and maintain reliable, scalable production systems across Citi’s tech estate.
• Architect and implement agentic AI workflows enabling autonomous decision-making with minimal human intervention.
• Apply prompt engineering to embed LLMs into operational tooling and incident response.
• Lead AI-native observability to detect anomalies and automate remediation before user impact.
• Write clean, tested, and well-documented code; promote best practices like code reviews and TDD.
• Drive Continuous Delivery and Automation via RCAs, knowledge mgmt, performance tuning, and training.
• Manage CI/CD, Infrastructure-as-Code tooling, and GitOps for scalable, safe delivery.
• Collaborate with platform, data, and product teams to embed AI in the production lifecycle.
• Implement Agile methodologies (SCRUM or Kanban) and align with organisational processes.
• Operate in a regulated financial setting, understanding compliance and data handling implications.
• Coach and mentor on AI engineering practices and agentic architectures; promote learning and excellence.
• Communicate progress clearly to stakeholders and foster transparency and innovation.

Key requirements

• Demonstrable software or production engineering experience with high business impact
• Hands-on AI/ML engineering experience including LLM APIs, embeddings, and vector databases
• Proven prompt engineering skills for production use cases
• Experience designing/deploying agentic systems using LangChain/LangGraph/AutoGen/CrewAI (or equivalents)
• Strong SDLC, GitOps, and DevSecOps understanding
• Knowledge of CS concepts (networks, OS, virtualization, containers)
• Polyglot full-stack development and quick language uptake
• Experience deploying services at scale on Docker/Kubernetes; service mesh knowledge preferred
• Experience with CI/CD orchestration and IaC tools (Terraform, CloudFormation, Pulumi)
• Agile delivery experience (SCRUM/Kanban); middleware exposure (MQ, Kafka) desirable
• Observability stacks familiarity (Datadog, AppDynamics, Dynatrace) desirable
• Degree in CS/Math/Physics or related field desirable
• Senior stakeholder management experience
• Strong written and verbal communication; global/remote collaboration experience

• Excellent communication
• Collaboration across disciplines
• Teachability and willingness to learn
• Java, Python, Go (or equivalent)
• LLM APIs (OpenAI, Anthropic, Gemini) and embeddings
• LangChain, LangGraph, AutoGen, CrewAI (or equivalents)

…