GenAI Guardrails & Observability Engineer - OpenShift

Bank of America; Pennington (NJ); On-site; USD 103,000 - 180,000; Full time

Benefits offered by this job

Annual discretionary bonus

Industry-leading benefits

Job summary

Bank of America in New Jersey is seeking an experienced software engineer to deploy, operate, and support third‑party GenAI applications in an on‑prem OpenShift platform, including Galileo and LangSmith.

You will monitor guardrails, observability, and model evaluations for LLM/RAG systems, build dashboards, and lead cross‑functional initiatives while mentoring engineers in a fast-paced enterprise setting. Experience with OpenShift/Kubernetes, Python ML libraries, and OpenAI APIs is valued.

Qualifications

• Bachelor's or Master's degree in Computer Science, Data Science, MIS, or related field, or equivalent experience.
• 10+ years of software engineering experience building and scaling production applications.
• Strong Python development experience, incl. ML/data libraries.
• Hands-on OpenShift or Kubernetes in production environments.
• Experience deploying and supporting LLM or RAG-based applications.
• Understanding AI risks, governance, and guardrails.
• Experience with OpenAI APIs, LangChain, and GenAI frameworks.
• Experience in Agile environments and leadership.

Responsibilities

• Develop, deploy, and operate vendor applications in an on-premises OpenShift environment.
• Build and maintain APIs and integrations connecting guardrail and observability platforms to GenAI apps.
• Monitor performance, safety, latency, throughput, and guardrail metrics.
• Implement and maintain model-based evaluation workflows, annotation, and calibration processes.
• Use LangChain for guardrail checks, prompt management, and scoring workflows.
• Build observability dashboards and alerting solutions.
• Troubleshoot production issues and optimize scalability and resilience.
• Lead and mentor engineers, driving cross-functional initiatives and best practices.

Skills

GenAI operations

OpenShift

Kubernetes

LangChain

Observability

Guardrails

Python

Vendor integrations

Lead mentoring

Agile

Education

Bachelor's or Master's in CS/DS/MIS or related

Tools

OpenShift

Kubernetes

LangSmith

Galileo

LangChain

Splunk

OpenAI APIs

Job description

Bank of America in New Jersey is seeking an experienced software engineer to deploy, operate, and support third‑party GenAI applications in an on‑prem OpenShift platform, including Galileo and LangSmith.

You will monitor guardrails, observability, and model evaluations for LLM/RAG systems, build dashboards, and lead cross‑functional initiatives while mentoring engineers in a fast-paced enterprise setting. Experience with OpenShift/Kubernetes, Python ML libraries, and OpenAI APIs is valued.