Senior AI Guardrails & Observability Engineer

Bank of America

Pennington (AL)

On-site

USD 122,000 - 200,000

Full time

14 days+

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Discretionary incentive eligible

Industry-leading benefits

Paid time off

Job summary

Bank of America is seeking a senior software engineer to own and operate vendor AI-enabled applications on an on‑prem OpenShift platform. You will build APIs, ensure governance, and deliver scalable, observable solutions across enterprise AI agent systems.

You will lead cross‑functional initiatives, mentor engineers, and drive delivery commitments in a fast-paced, regulated environment with aggressive timelines. A strong focus on security, performance, and reliability is essential.

Qualifications

Bachelor’s or Master’s degree in Computer Science, Data Science, MIS, or related field; or equivalent practical experience.
13+ years of software engineering experience designing, building, and scaling production applications.
Strong proficiency in Python and data/ML libraries; hands-on with containerized workloads on OpenShift or Kubernetes.
Proven experience bringing LLM- or RAG-based applications to production at scale.
Experience building APIs and integrating third-party platforms.
Working experience with AI Agents development, including OpenAI APIs and LangChain.
Understanding of inference performance optimization; applying guardrails, observability, and model-based evaluation.
Experience delivering within Agile environments against firm timelines.
Proven ability to influence outcomes in a matrixed org without direct authority.

Responsibilities

Own delivery of critical vendor applications from build through production support in a fully on‑premises OpenShift environment.
Build and maintain APIs and integrations connecting guardrail and observability platforms to AI Agent solutions.
Establish workflows to monitor performance, safety, latency, throughput, quality, and guardrail effectiveness.
Implement model evaluation frameworks using LLM-as-a-Judge and specialized language models.
Develop observability dashboards and alerting with Grafana; produce audit-friendly evidence.
Troubleshoot production issues and improve scalability, resilience, and availability.
Lead and mentor engineers; establish development standards and drive process improvements.
Provide input into financial planning, budgeting, and forecasting for supported apps.