GenAI Guardrails & Observability Engineer

NCSL International; Charlotte, Northern (NC, KY); Hybrid; USD 140,000 - 210,000; Full time

Job summary

Bank of America is seeking an experienced engineer to run GenAI operations, deploying and supporting third-party applications in an on-premises OpenShift platform. You will monitor guardrails, observability, and model evaluation workflows across enterprise AI apps such as LLMs and RAG-based systems.

The role requires deep Python development, production OpenShift/Kubernetes experience, and the ability to lead cross-functional initiatives, mentor engineers, and deliver scalable AI governance

Qualifications

• Bachelor's or Master's degree in CS/DS/MIS or related field, or equivalent experience.
• 10+ years of software engineering experience.
• Strong Python development experience with ML/data libraries.
• Hands-on OpenShift/Kubernetes in production.
• Experience deploying LLM/RAG-based applications.
• Understanding of AI risks, agent behavior, and governance frameworks.
• Experience integrating third-party platforms and supporting vendor applications.
• Experience with OpenAI APIs, LangChain, and GenAI development frameworks.
• Knowledge of inference performance optimization and scalability.
• Experience implementing AI guardrails, observability, and model evaluation processes.
• Experience working in Agile environments.
• Proven ability to lead technical initiatives and influence stakeholders.
• Strong analytical, problem-solving, and communication skills.

Responsibilities

• Develop, deploy, and operate vendor applications in an on-premises OpenShift environment.
• Build and maintain APIs connecting guardrail and observability platforms to enterprise GenAI applications.
• Monitor application performance, safety, latency, throughput, and guardrail metrics.
• Implement and maintain model-based evaluation workflows, including annotation and calibration processes.
• Utilize orchestration frameworks such as LangChain for guardrail checks, prompt management, and scoring workflows.
• Build observability dashboards and alerting solutions.
• Troubleshoot production issues and optimize performance, scalability, and resilience.
• Contribute to emerging AI evaluation and guardrail strategies for agentic workflows.
• Lead and mentor engineers while driving cross-functional initiatives and engineering best practices.

Skills

Python development

OpenShift/Kubernetes

LangChain

AI governance

Agile

Leadership

Communication

Problem solving

Education

Bachelor's or Master's in CS/DS/MIS or equivalent

Tools

OpenShift

Kubernetes

LangChain

Galileo

LangSmith

OpenAI APIs

Splunk

Job description

Bank of America is seeking an experienced engineer to run GenAI operations, deploying and supporting third-party applications in an on-premises OpenShift platform. You will monitor guardrails, observability, and model evaluation workflows across enterprise AI apps such as LLMs and RAG-based systems.

The role requires deep Python development, production OpenShift/Kubernetes experience, and the ability to lead cross-functional initiatives, mentor engineers, and deliver scalable AI governance