GenAI Guardrails & Observability Engineer

NCSL International; Richmond (VA); On-site; USD 103,000 - 180,000; Full time

Job summary

Bank of America in the United States is seeking a Senior GenAI Operations Engineer to deploy, monitor, and optimize on-premises OpenShift-based vendor platforms for enterprise AI workloads.

You will implement guardrails, observability, and model evaluation workflows across LLM/RAG apps, integrate LangChain and other tooling, and lead cross-functional initiatives with a pragmatic build-and-operate mindset.

Qualifications

• Bachelor's or Master's degree in Computer Science, Data Science, MIS, or related field, or equivalent experience.
• 10+ years of software engineering experience building and scaling production applications.
• Strong Python development experience with ML/data libraries.
• Hands-on OpenShift or Kubernetes in production environments.
• Experience deploying and supporting LLM/RAG-based applications.
• Understanding of AI risks, agent behavior, and governance frameworks.
• Experience integrating third-party platforms and supporting vendor applications.
• Experience with OpenAI APIs, LangChain, and GenAI development frameworks.
• Knowledge of inference performance optimization and scalability.
• Experience implementing AI guardrails, observability, and model evaluation processes.
• Experience working in Agile environments.
• Proven ability to lead technical initiatives and influence stakeholders.
• Strong analytical, problem-solving, and communication skills.

Responsibilities

• Develop, deploy, and operate vendor applications in an on-premises OpenShift environment.
• Build and maintain APIs and integrations connecting guardrail and observability platforms to GenAI apps.
• Monitor application performance, safety, latency, throughput, and guardrail metrics.
• Implement and maintain model-based evaluation workflows, including annotation and calibration processes.
• Utilize orchestration frameworks such as LangChain for guardrail checks, prompt management, and scoring workflows.
• Build observability dashboards and alerting solutions.
• Troubleshoot production issues and optimize performance, scalability, and resilience.
• Contribute to evolving AI evaluation and guardrail strategies for agentic workflows.
• Lead and mentor engineers while driving cross-functional initiatives and engineering best practices.

Skills

Application Development

Automation

Influence

Solution Design

Technical Strategy Development

Architecture

Business Acumen

DevOps Practices

Result Orientation

Solution Delivery Process

Analytical Thinking

Collaboration

Data Management

Risk Management

Test Engineering

Education

Bachelor's or Master's degree in Computer Science, Data Science, MIS, or related field, or equivalent experience.

Tools

OpenShift

Kubernetes

LangChain

LangSmith

Galileo

Splunk

LlamaIndex

Haystack

OpenAI APIs

Job description

Bank of America in the United States is seeking a Senior GenAI Operations Engineer to deploy, monitor, and optimize on-premises OpenShift-based vendor platforms for enterprise AI workloads.

You will implement guardrails, observability, and model evaluation workflows across LLM/RAG apps, integrate LangChain and other tooling, and lead cross-functional initiatives with a pragmatic build-and-operate mindset.