Lead post-incident reviews (RCAs/post-mortems), identify recurring failure patterns, and partner with development teams on root cause remediation to eliminate recurrence and reduce technical debt
Maintain strong production governance — access controls, operational procedures, runbooks, documentation, audit readiness, MEPC execution, and operational review processes
Design, build, and maintain production-grade software (primarily Python) for operational tooling, self-healing workflows, and automated runbooks that reduce manual toil and MTTR
Contribute code changes directly to Rates applications to fix production defects, improve performance, and harden resiliency, following the full SDLC - design, automated testing, peer review, and controlled release
Build reusable engineering components, APIs, and integrations (job schedulers, ticketing, messaging, data stores) that scale improvements across the CIB application portfolio
Design and deliver AI/ML-based early-detection capabilities - anomaly detection on batch timings, job volumes, risk outputs, and market data, with predictive alerting on SLA breaches before business impact
Define and improve SLIs, SLOs, error budgets, and reliability metrics for the Rates application suite, reporting production health to senior technology and business leadership
Design and deploy monitoring, telemetry, dashboards, and alerting (Geneos, OpenTelemetry, Grafana, or equivalent)
Drive capacity planning, performance profiling, resiliency testing, and BCP/failover readiness across multi-region deployments
Represent Production Management in architecture and production readiness reviews for new features, platform changes, and cloud migration
Build and maintain strong relationships with Application Development, DevOps, Front Office business users, and GTI/Infrastructure teams
Communicate effectively with all levels of management - from technical deep-dives to executive-level summaries of production health and automation impact
Technical Expertise (Required)
Experience: 7+ years' experience in front-office production support, SRE, or production engineering within Investment Banking, preferably supporting Fixed Income Rates trading applications (pricing, risk, trade management, EOD/batch processing)
Software Engineering: Formal training or certification in software engineering concepts; strong Python skills at production-code level - object-oriented design, testing, debugging complex distributed systems, and building automation tooling within a full SDLC with code review and CI/CD
AI & Automation: Hands-on experience applying AI/ML or LLM-based tooling to operational problems (anomaly detection, triage, RCA, automated remediation, reporting) and leading effective use of AI-assisted development tools, with a strong understanding of responsible AI - data sensitivity, guardrails, and validation of outputs
Monitoring & Observability: Extensive experience with Geneos and/or Grafana, Splunk, Dynatrace, OpenTelemetry, log aggregation, and alerting frameworks
Databases: Strong SQL (Oracle, Postgres) and query performance tuning in high-throughput environments
Platform Knowledge: Linux/UNIX (RHEL), shell scripting (bash/ksh), job scheduling systems, and batch orchestration
Incident Management: Proven experience leading complex P1 incidents, managing war rooms, problem management, and producing actionable RCAs that eliminate recurrence
Technical Expertise (Preferred)
Knowledge of Rates products (Swaps, Options, Inflation, Bonds/Futures, Repo/Securities Lending) and the front-to-back trade lifecycle
Experience building agentic AI workflows, prompt engineering, or integrating LLMs with internal tooling (e.g. MCP servers, knowledge graphs, retrieval-augmented generation)
Hands-on experience with public cloud (AWS), infrastructure-as-code (Terraform, CloudFormation), and containerisation (OpenShift, Kubernetes, Docker)
Experience with time-series/statistical analysis and data libraries for building data-driven detection and dashboards
Experience with capacity management, performance profiling, and memory analysis in Python/Java environments
General Competencies
Proven ability to lead under pressure - managing multiple critical incidents simultaneously while maintaining clear communication
Engineering mindset - automates repeatable work, measures outcomes, and treats operational problems as software problems
Strategic thinker with the ability to balance immediate production needs against long-term platform and automation improvements
Strong ownership mentality with a track record of driving initiatives from conception to measurable outcomes (e.g. reduced MTTR, fewer incidents, improved SLA adherence)
Excellent communication skills (verbal and written) - able to translate complex technical issues into business impact language
Curiosity and continuous learning, particularly in emerging AI technologies, applied with sound judgement on risk and control