Lead AI Monitoring & Guardrails Engineer (On-Prem OpenShift)

Bank of America; Pennington (NJ); On-site; USD 180,000 - 240,000; Full time

Job summary

Bank of America seeks a senior engineer to own the end-to-end delivery of on-prem vendor AI-related applications within OpenShift. You will build APIs, guardrails, and observability for enterprise AI Agent solutions, ensuring audit-ready, scalable production services.

You will lead cross-functional teams, drive architectural standards, and contribute to budgeting and planning while maintaining a sense of urgency and solid operational judgment.

Qualifications

• Bachelor's or Master's degree in CS/DS/MIS or equivalent experience.
• 13+ years of software engineering designing, building, and scaling production apps.
• Strong Python, data/ML libraries; hands-on OpenShift/Kubernetes experience.
• Proven production experience with LLM-/RAG-based apps and scaling.
• Experience building APIs and integrating third-party platforms.
• Experience with AI Agents, OpenAI APIs, LangChain.
• Understanding of inference performance optimization and guardrails.

Responsibilities

• Own delivery of critical vendor apps in a fully on-prem OpenShift environment, hitting milestones.
• Build APIs and integrations linking guardrail and observability to AI Agent solutions.
• Establish workflows monitoring performance, latency, throughput, quality, and guardrail efficacy.
• Implement model evaluation frameworks using LLM-as-a-Judge and specialized models.
• Develop observability dashboards and alerts with Grafana; ensure audit readiness.
• Troubleshoot production issues, improve scalability and resilience for critical services.
• Contribute to evaluation methodologies and guardrails for AI systems.
• Lead engineers, set development standards, and drive cross-functional improvements.
• Contribute to budgeting, forecasting, and management of supported apps.

Skills

Automation

Influence

Result Orientation

Stakeholder Management

Technical Strategy Development

Application Development

Architecture

Risk Management

Solution Design

Agile Practices

Analytical Thinking

Collaboration

Data Management

Solution Delivery Process

Education

Bachelor's or Master's degree in Computer Science, Data Science, MIS, a related field, or equivalent practical experience

Tools

OpenShift

Kubernetes

OpenAI APIs

LangChain

Grafana

Splunk

LangSmith

Job description

Bank of America seeks a senior engineer to own the end-to-end delivery of on-prem vendor AI-related applications within OpenShift. You will build APIs, guardrails, and observability for enterprise AI Agent solutions, ensuring audit-ready, scalable production services.

You will lead cross-functional teams, drive architectural standards, and contribute to budgeting and planning while maintaining a sense of urgency and solid operational judgment.