Senior AI Observability Engineer — On-Prem OpenShift

Bank of America

Town of Charlotte (NY)

On-site

USD 122,000 - 200,000

Full time

13 days ago

Application generator

Turn this role into an interview — a resume and cover letter built around what this employer wants.

Get past ATS filters

Benefits offered by this job

Industry-leading benefits

Paid time off

Discretionary incentive plan

Job summary

Bank of America in Jersey City, NJ seeks an experienced software engineer to own delivery of critical vendor applications on an on-prem OpenShift platform. You will develop APIs, guardrails, and observability for AI Agent systems, ensuring production reliability.

Strong Python, OpenShift/Kubernetes, and LangChain experience are essential. You will collaborate across teams to implement scalable solutions, drive best practices, and align with risk/compliance requirements in a fast-paced, governed

Qualifications

13+ years of software engineering experience designing, building, and scaling production applications.
Strong proficiency in Python and ML libraries.
Experience deploying and operating containerized workloads on OpenShift or Kubernetes.
Experience taking LLM- or RAG-based applications into production and scaling them.
Experience building APIs and integrating third-party platforms.
Experience with AI Agents development, including OpenAI APIs and LangChain.
Experience delivering in Agile environments and meeting firm timelines.
Ability to influence outcomes in a matrixed organization without direct authority.
Strong problem-solving and stakeholder management.

Responsibilities

Own delivery of critical vendor applications from build through production support in a fully on-premises OpenShift environment.
Build and maintain APIs and integrations connecting guardrail and observability platforms to enterprise AI Agent solutions.
Establish and manage workflows that monitor performance, safety, latency, throughput, quality, and guardrail effectiveness.
Implement and maintain model evaluation frameworks using LLM-as-a-Judge and specialized language models.
Develop observability dashboards and alerting using Grafana, producing audit-friendly evidence and traceability.
Troubleshoot production issues and improve scalability, resilience, and operational efficiency.
Contribute to evaluation methodologies and guardrail controls for agentic and multi-step AI systems.
Lead and mentor engineers, establish development standards, and drive process improvements across cross-functional delivery.
Provide input into financial planning, budget tracking, forecasting, and management of supported applications and services.