Senior AI Monitoring & Controls Engineer (On-Prem OpenShift)

Bank of America; Virginia; On-site; USD 122,000 - 200,000; Full time

Benefits offered by this job

Discretionary incentive eligible

Job summary

Bank of America is seeking an experienced senior software engineer to own end-to-end delivery of on-prem OpenShift-based vendor applications supporting AI monitoring, guardrails, and governance. You will deploy, scale containers, and provide production support to ensure reliability and audit readiness.

The role emphasizes rapid tool adoption, cross-functional leadership, and building scalable platforms for model evaluation, observability, and risk management within a regulated financial services

Qualifications

• Bachelor's or Master’s degree in CS/DS/MIS or equivalent practical experience.
• 13+ years of software engineering experience building scalable production apps.
• Proficient in Python with experience deploying containerized workloads on OpenShift or Kubernetes.
• Experience bringing LLM/RAG-based apps to production at scale.
• Experience building APIs and integrating third-party platforms.
• Experience with AI Agents development, OpenAI APIs, and LangChain.
• Strong Agile delivery experience and stakeholder management.

Responsibilities

• Own delivery of critical vendor applications from build through production support in a fully on-premises OpenShift environment, meeting milestones and quality expectations.
• Build and maintain APIs and integrations connecting vendor guardrail and observability platforms to enterprise AI Agent solutions.
• Establish and manage workflows that monitor performance, safety, latency, throughput, quality, and guardrail effectiveness.
• Implement and maintain model evaluation frameworks using LLM-as-a-Judge and specialized language models.
• Develop observability dashboards and alerting using Grafana or similar tools, producing audit-friendly evidence and traceability.
• Troubleshoot production issues and improve scalability, resilience, and operational efficiency.
• Contribute to evaluation methodologies and guardrail controls for agentic and multi-step AI systems.
• Lead and mentor engineers, establish development standards, and drive process improvements across cross-functional delivery.
• Provide input into financial planning, budget tracking, forecasting, and management of supported applications and services.

Skills

Automation

Influence

Result Orientation

Stakeholder Management

Technical Strategy Development

Application Development

Architecture

Business Acumen

Risk Management

Solution Design

Agile Practices

Analytical Thinking

Collaboration

Data Management

Solution Delivery Process

Education

Bachelor's or Master’s degree in Computer Science, Data Science, MIS, or related field

Tools

OpenShift

Kubernetes

Grafana

LangChain

OpenAI APIs

Splunk

LlamaIndex

Haystack

Job description

Bank of America is seeking an experienced senior software engineer to own end-to-end delivery of on-prem OpenShift-based vendor applications supporting AI monitoring, guardrails, and governance. You will deploy, scale containers, and provide production support to ensure reliability and audit readiness.

The role emphasizes rapid tool adoption, cross-functional leadership, and building scalable platforms for model evaluation, observability, and risk management within a regulated financial services