Senior AI Monitoring & Controls Engineer

Bank of America; Jacksonville (FL); On-site; USD 150,000 - 210,000; Full time

Job summary

Bank of America is seeking an experienced software engineer to own delivery of critical AI agent operations on a fully on‑prem OpenShift platform. You will develop and integrate components, deploy containerized services, and provide production support for guardrails and observability across enterprise AI solutions.

The role requires strong Python, ML libraries, and hands‑on OpenShift/Kubernetes experience, with a track record of productionizing LLM‑based applications and leading cross‑functional

Qualifications

• Bachelor’s or Master’s degree in CS/DS/MIS or equivalent practical experience.
• 13+ years of software engineering experience.
• Strong Python skills and ML libraries; hands‑on with containerized workloads on OpenShift/Kubernetes.
• Experience taking LLM/RAG‑based apps into production and maintaining scale.
• Experience building APIs and integrating third‑party platforms.

Responsibilities

• Own delivery of critical vendor applications from build through production support in a fully on‑premises OpenShift environment.
• Build and maintain APIs and integrations connecting vendor guardrail and observability platforms to enterprise AI Agent solutions.
• Establish and manage workflows that monitor performance, safety, latency, throughput, quality, and guardrail effectiveness.
• Implement and maintain model evaluation frameworks using LLM‑as‑a‑Judge and language models.
• Develop observability dashboards and alerting using Grafana or similar tools; produce audit‑friendly evidence and traceability.
• Troubleshoot production issues and improve scalability, resilience, and operational efficiency.

Skills

Automation

Influence

Result Orientation

Stakeholder Management

Technical Strategy

Agile Practices

Analytical Thinking

Collaboration

Education

Bachelor's or Master's degree in Computer Science, Data Science, MIS, or related field

Tools

OpenShift

Kubernetes

Grafana

LangChain

OpenAI APIs

LangSmith

Splunk

Job description

Bank of America is seeking an experienced software engineer to own delivery of critical AI agent operations on a fully on‑prem OpenShift platform. You will develop and integrate components, deploy containerized services, and provide production support for guardrails and observability across enterprise AI solutions.

The role requires strong Python, ML libraries, and hands‑on OpenShift/Kubernetes experience, with a track record of productionizing LLM‑based applications and leading cross‑functional