GenAI Guardrails & Observability Engineer

Bank of America; Jersey City (NJ); On-site; USD 103,000 - 180,000; Full time

Benefits offered by this job

Discretionary incentive plan

Industry-leading benefits

Paid time off

Job summary

Bank of America in Jersey City is seeking an experienced software engineer to design, deploy, and operate third-party GenAI applications on an on-prem OpenShift platform. You will work with vendor tools like Galileo and LangSmith, implement guardrails, observability, and model evaluation workflows across the enterprise.

The role demands deep Python experience, Kubernetes/OpenShift production chops, and the ability to lead cross-functional initiatives while mentoring engineers.

Qualifications

• Bachelor’s or Master’s degree in CS, Data Science, MIS, or equivalent.
• 10+ years of software engineering experience building production applications.
• Strong Python development experience with ML and data libraries.
• Hands-on experience with OpenShift or Kubernetes in production environments.
• Experience deploying and supporting LLM or RAG-based applications.
• Understanding of AI risks, agent behavior, and governance frameworks.
• Experience integrating third-party platforms and supporting vendor applications.
• Experience with OpenAI APIs, LangChain, and GenAI development frameworks.
• Knowledge of inference performance optimization and scalability.
• Experience implementing AI guardrails, observability, and model evaluation processes.
• Experience working in Agile environments.
• Proven ability to lead technical initiatives and influence stakeholders.
• Strong analytical, problem-solving, and communication skills.

Responsibilities

• Develop, deploy, and operate vendor applications in an on-premises OpenShift environment.
• Build and maintain APIs and integrations connecting vendor guardrail and observability platforms to enterprise GenAI applications.
• Monitor application performance, safety, latency, throughput, and guardrail metrics.
• Implement and maintain model-based evaluation workflows, including annotation and calibration processes.
• Utilize orchestration frameworks such as LangChain for guardrail checks, prompt management, and scoring workflows.
• Build observability dashboards and alerting solutions.
• Troubleshoot production issues and optimize performance, scalability, and resilience.
• Contribute to emerging AI evaluation and guardrail strategies for agentic workflows.
• Lead and mentor engineers while driving cross-functional initiatives and engineering best practices.

Skills

10+ years software engineering

Python development

OpenShift / Kubernetes

LLM / RAG deployment

OpenAI APIs, LangChain

AI guardrails & governance

Agile / cross-functional leadership

Mentoring engineers

Education

Bachelor's or Master's in CS/Data Science/MIS or equivalent

Tools

OpenShift

Kubernetes

LangChain

Galileo

LangSmith

Splunk

GenAI frameworks

LlamaIndex / Haystack

Job description

Bank of America in Jersey City is seeking an experienced software engineer to design, deploy, and operate third-party GenAI applications on an on-prem OpenShift platform. You will work with vendor tools like Galileo and LangSmith, implement guardrails, observability, and model evaluation workflows across the enterprise.

The role demands deep Python experience, Kubernetes/OpenShift production chops, and the ability to lead cross-functional initiatives while mentoring engineers.