GenAI Guardrails & Observability Engineer

Bank of America; Charlotte (NC); On-site; USD 103,000 - 180,000; Full time

Job summary

Bank of America seeks an experienced software engineer to own GenAI operations across enterprise applications on an on‑premises OpenShift platform. You will deploy, monitor, and optimize LLM/RAG-based solutions with vendor tools like Galileo and LangSmith, focusing on guardrails, evaluation workflows, and observability.

You will work with cross‑functional teams to operationalize model-based evaluation, calibrate workflows, and deliver robust guardrail strategies in a fast-paced environment.

Qualifications

• Bachelor's or Master's degree in Computer Science, Data Science, MIS, or related field, or equivalent experience.
• 10+ years of software engineering experience building and scaling production applications.
• Strong Python development experience, including common ML and data libraries.
• Hands-on experience with OpenShift or Kubernetes in production environments.
• Experience deploying and supporting LLM or RAG-based applications.
• Understanding of AI risks, agent behavior, and governance frameworks.
• Experience integrating third-party platforms and supporting vendor applications.
• Experience with OpenAI APIs, LangChain, and GenAI development frameworks.
• Knowledge of inference performance optimization and scalability.
• Experience implementing AI guardrails, observability, and model evaluation processes.
• Experience working in Agile environments.
• Proven ability to lead technical initiatives and influence stakeholders.
• Strong analytical, problem-solving, and communication skills.

Responsibilities

• Develop, deploy, and operate vendor applications in an on-premises OpenShift environment.
• Build and maintain APIs and integrations connecting vendor guardrail and observability platforms to enterprise GenAI applications.
• Monitor application performance, safety, latency, throughput, and guardrail metrics.
• Implement and maintain model-based evaluation workflows, including annotation and calibration processes.
• Utilize orchestration frameworks such as LangChain for guardrail checks, prompt management, and scoring workflows.
• Build observability dashboards and alerting solutions.
• Troubleshoot production issues and optimize performance, scalability, and resilience.
• Lead cross-functional initiatives and engineering best practices.

Skills

Application Development

Automation

Influence

Solution Design

Technical Strategy Development

Architecture

Business Acumen

DevOps Practices

Result Orientation

Solution Delivery Process

Analytical Thinking

Collaboration

Data Management

Risk Management

Test Engineering

Education

Bachelor's or Master's degree in Computer Science, Data Science, MIS, or related field

Tools

OpenShift

Kubernetes

LangChain

Galileo

LangSmith

Splunk

Job description

Bank of America seeks an experienced software engineer to own GenAI operations across enterprise applications on an on‑premises OpenShift platform. You will deploy, monitor, and optimize LLM/RAG-based solutions with vendor tools like Galileo and LangSmith, focusing on guardrails, evaluation workflows, and observability.

You will work with cross‑functional teams to operationalize model-based evaluation, calibrate workflows, and deliver robust guardrail strategies in a fast-paced environment.