GenAI Guardrails & Observability Engineer

NCSL International; Plano (TX); On-site; USD 103,000 - 180,000; Full time

Benefits offered by this job

Benefits eligible

Discretionary incentive plan

Job summary

Bank of America is seeking an experienced Software Engineer to manage GenAI operations on an on-prem OpenShift platform. The role focuses on deploying vendor platforms like Galileo and LangSmith, ensuring model evaluation workflows are maintained, and monitoring guardrails across enterprise applications.

The engineer will lead cross-functional initiatives, implement observability, and mentor team members in a fast-paced environment with strong emphasis on governance and safety.

Qualifications

• 10+ years of software engineering experience building and scaling production applications.
• Strong Python development experience with ML/data libraries.
• Hands-on experience with OpenShift or Kubernetes in production.
• Experience deploying and supporting LLM or RAG-based applications.
• Understanding AI risks, governance frameworks, and guardrails.
• Experience with OpenAI APIs, LangChain, and GenAI frameworks.
• Experience leading technical initiatives in Agile environments.

Responsibilities

• Develop, deploy, and operate vendor applications in an on-premises OpenShift environment.
• Build and maintain APIs and integrations for guardrail and observability platforms.
• Monitor performance, safety, latency, throughput, and guardrail metrics.
• Implement model-based evaluation workflows and calibration processes.
• Utilize LangChain for guardrail checks, prompt management, and scoring workflows.
• Build observability dashboards and alerting solutions.
• Troubleshoot production issues and optimize performance, scalability, and resilience.
• Lead cross-functional initiatives and mentor engineers.

Skills

Application Development

Automation

Influence

Solution Design

Technical Strategy Development

Architecture

DevOps Practices

Result Orientation

Analytical Thinking

Collaboration

Data Management

Risk Management

Test Engineering

Education

Bachelor's degree or equivalent experience

Tools

OpenShift

Kubernetes

OpenAI APIs

LangChain

Galileo

LangSmith

Splunk

Job description

Bank of America is seeking an experienced Software Engineer to manage GenAI operations on an on-prem OpenShift platform. The role focuses on deploying vendor platforms like Galileo and LangSmith, ensuring model evaluation workflows are maintained, and monitoring guardrails across enterprise applications.

The engineer will lead cross-functional initiatives, implement observability, and mentor team members in a fast-paced environment with strong emphasis on governance and safety.