Senior AI Monitoring & Control Engineer

Bank of America; Plano (TX); On-site; USD 122,000 - 200,000; Full time

Job summary

Bank of America is seeking an experienced engineer to own delivery of critical vendor applications on an on-prem OpenShift platform, delivering production support for AI monitoring, controls, and governance.

The role requires 13+ years of software engineering experience, Python proficiency, and proven success deploying LLM/RAG-based solutions at scale in a regulated enterprise. You will build APIs, ensure observability, and lead cross-functional delivery initiatives.

Qualifications

• 13+ years of software engineering experience designing, building, and scaling production applications.
• Strong proficiency in Python, including common data and ML libraries. Hands-on experience deploying and operating containerized workloads on OpenShift or Kubernetes.
• Proven experience taking LLM- or RAG-based applications into production and maintaining them at scale.
• Experience building APIs and integrating third-party platforms.
• Working experience with AI Agents development, including OpenAI APIs and LangChain.
• Practical understanding of inference performance optimization. Experience applying guardrails, observability, and model-based evaluation to AI applications.
• Experience delivering within Agile and Agile-at-scale environments against firm timelines.
• Proven ability to influence outcomes and meet delivery commitments in a matrixed organization without direct authority. Strong problem-solving, communication, and stakeholder management skills.

Responsibilities

• Own delivery of critical vendor applications from build through production support in a fully on-premises OpenShift environment, meeting committed milestones and quality expectations.
• Build and maintain APIs and integrations connecting vendor guardrail and observability platforms to enterprise AI Agent solutions.
• Establish and manage workflows that monitor performance, safety, latency, throughput, quality, and guardrail effectiveness in support of AI risk and control objectives.
• Implement and maintain model evaluation frameworks using LLM-as-a-Judge and specialized language models.
• Develop observability dashboards and alerting using Grafana or similar tools, producing audit-friendly evidence and traceability.
• Troubleshoot production issues and improve scalability, resilience, and operational efficiency to sustain availability for critical business services.
• Contribute to evaluation methodologies and guardrail controls for agentic and multi-step AI systems.
• Lead and mentor engineers, establish development standards, and drive process improvements across cross-functional delivery.
• Provide input into financial planning, budget tracking, forecasting, and management of supported applications and services.

Skills

Automation

Influence

Result Orientation

Stakeholder Management

Technical Strategy Development

Application Development

Architecture

Business Acumen

Risk Management

Solution Design

Agile Practices

Analytical Thinking

Collaboration

Data Management

Solution Delivery Process

Education

CS/DS/MIS degree or related field

Job description

Bank of America is seeking an experienced engineer to own delivery of critical vendor applications on an on-prem OpenShift platform, delivering production support for AI monitoring, controls, and governance.

The role requires 13+ years of software engineering experience, Python proficiency, and proven success deploying LLM/RAG-based solutions at scale in a regulated enterprise. You will build APIs, ensure observability, and lead cross-functional delivery initiatives.