AI Monitoring & Guardrails Engineer — On-Prem OpenShift

Bank of America; Richmond (VA); On-site; USD 180,000 - 240,000; Full time

Job summary

Bank of America seeks an experienced engineer to own critical vendor AI-powered applications on an on-prem OpenShift platform. You will develop APIs, ensure observability and governance, and lead cross-functional initiatives to deliver reliable, scalable AI agent solutions.

The role requires deep Python and ML library experience, hands-on OpenShift/Kubernetes, and a track record of bringing LLM-based apps into production.

Qualifications

• Bachelor's or Master’s in CS, Data Science, MIS, or equivalent practical experience.
• 13+ years of software engineering experience designing, building, and scaling production applications.
• Strong Python proficiency with ML libraries; hands-on OpenShift/Kubernetes.
• Proven experience taking LLM- or RAG-based applications into production and scaling them.
• Experience building APIs and integrating third-party platforms.
• Experience with AI Agents development, including OpenAI APIs and LangChain.
• Understanding of inference performance optimization and guardrails for AI apps.
• Experience delivering in Agile environments and influencing outcomes without direct authority.

Responsibilities

• Own delivery of critical vendor applications from build through production support in an on-prem OpenShift environment.
• Build and maintain APIs connecting guardrail and observability platforms to AI Agent solutions.
• Establish workflows to monitor performance, latency, throughput, and guardrail effectiveness.
• Implement model evaluation frameworks using LLM-as-a-Judge and specialized language models.
• Develop observability dashboards and alerting with Grafana for audit-friendly traceability.
• Troubleshoot production issues and improve scalability and resilience.
• Contribute to evaluation methodologies and guardrail controls for AI systems.
• Lead and mentor engineers and drive cross-functional process improvements.
• Provide input into budgeting, forecasting, and management of applications.

Skills

Python

OpenShift

Kubernetes

LLM production

APIs

LangChain

Guardrails

Agile

Stakeholder mgmt

Education

Bachelor's or Master’s in CS/DS/MIS

Tools

Grafana

OpenAI API

LangChain

Job description

Bank of America seeks an experienced engineer to own critical vendor AI-powered applications on an on-prem OpenShift platform. You will develop APIs, ensure observability and governance, and lead cross-functional initiatives to deliver reliable, scalable AI agent solutions.

The role requires deep Python and ML library experience, hands-on OpenShift/Kubernetes, and a track record of bringing LLM-based apps into production.