GenAI Observability & Governance Engineer (On-Prem)

National Black MBA Association; Jersey City, Northern (NJ, KY); On-site; USD 103,000 - 180,000; Full time

Benefits offered by this job

Industry-leading benefits

Annual discretionary plan

Competitive compensation

Job summary

Bank of America is seeking an experienced engineer to own GenAI operations in an on-prem OpenShift environment, focusing on monitoring, guardrails, and governance for LLM/RAG workflows. You will integrate LangChain-based pipelines, surface metrics via dashboards, and drive cross-functional delivery across teams.

The role emphasizes practical build-and-operate mindset, rapid tool adoption, and leadership to advance enterprise AI initiatives with strong emphasis on safety and performance.

Qualifications

• Bachelor's or Master's degree in Computer Science, Data Science, MIS, related field, or equivalent experience.
• 10+ years of software engineering experience designing, building, and scaling production applications.
• Strong Python development experience, including data and machine learning libraries.
• Experience deploying and supporting containerized workloads on OpenShift or Kubernetes.
• Experience taking LLM or RAG-based applications into production.
• Understanding of AI risks, agent behavior, and governance frameworks.
• Experience building APIs and integrating third-party platforms.
• Experience with OpenAI APIs and orchestration frameworks such as LangChain.
• Understanding of inference performance optimization and observability.
• Experience implementing AI guardrails and model-based evaluation frameworks.
• Experience working in Agile environments.
• Demonstrated technical leadership and stakeholder influence.

Responsibilities

• Develop, deploy, and support vendor applications in a fully on-premises OpenShift environment.
• Build APIs and integrations connecting guardrail and observability tools with enterprise GenAI applications.
• Implement and maintain operational monitoring for application performance, safety, latency, and quality metrics.
• Develop and calibrate model-based evaluation workflows including annotation and feedback processes.
• Utilize orchestration frameworks such as LangChain for guardrails, prompt management, and scoring workflows.
• Create observability dashboards and alerting capabilities.
• Troubleshoot production issues and improve scalability, reliability, and performance.
• Support emerging agentic and multi-step AI workflows.
• Lead and mentor team members while driving engineering standards and cross-functional delivery.

Skills

Application Development

Automation

Influence

Solution Design

Technical Strategy Development

Architecture

Business Acumen

DevOps Practices

Result Orientation

Solution Delivery Process

Analytical Thinking

Collaboration

Data Management

Risk Management

Test Engineering

Education

Bachelor's or Master's degree in Computer Science, Data Science, MIS, related field, or equivalent experience

Tools

OpenShift

Kubernetes

LangChain

LangSmith

Galileo

Splunk

OpenAI APIs

LlamaIndex

Haystack

Job description

Bank of America is seeking an experienced engineer to own GenAI operations in an on-prem OpenShift environment, focusing on monitoring, guardrails, and governance for LLM/RAG workflows. You will integrate LangChain-based pipelines, surface metrics via dashboards, and drive cross-functional delivery across teams.

The role emphasizes practical build-and-operate mindset, rapid tool adoption, and leadership to advance enterprise AI initiatives with strong emphasis on safety and performance.