AI Engineer - Guardrails and Observability

Bank of America; Pennington (NJ); On-site; USD 103,000 - 180,000; Full time

Benefits offered by this job

Annual discretionary bonus

Industry-leading benefits

Job summary

Bank of America in New Jersey is seeking an experienced software engineer to deploy, operate, and support third‑party GenAI applications in an on‑prem OpenShift platform, including Galileo and LangSmith.

You will monitor guardrails, observability, and model evaluations for LLM/RAG systems, build dashboards, and lead cross‑functional initiatives while mentoring engineers in a fast-paced enterprise setting. Experience with OpenShift/Kubernetes, Python ML libraries, and OpenAI APIs is valued.

Qualifications

• Bachelor's or Master's degree in Computer Science, Data Science, MIS, or related field, or equivalent experience.
• 10+ years of software engineering experience building and scaling production applications.
• Strong Python development experience, incl. ML/data libraries.
• Hands-on OpenShift or Kubernetes in production environments.
• Experience deploying and supporting LLM or RAG-based applications.
• Understanding AI risks, governance, and guardrails.
• Experience with OpenAI APIs, LangChain, and GenAI frameworks.
• Experience in Agile environments and leadership.

Responsibilities

• Develop, deploy, and operate vendor applications in an on-premises OpenShift environment.
• Build and maintain APIs and integrations connecting guardrail and observability platforms to GenAI apps.
• Monitor performance, safety, latency, throughput, and guardrail metrics.
• Implement and maintain model-based evaluation workflows, annotation, and calibration processes.
• Use LangChain for guardrail checks, prompt management, and scoring workflows.
• Build observability dashboards and alerting solutions.
• Troubleshoot production issues and optimize scalability and resilience.
• Lead and mentor engineers, driving cross-functional initiatives and best practices.

Skills

GenAI operations

OpenShift

Kubernetes

LangChain

Observability

Guardrails

Python

Vendor integrations

Lead mentoring

Agile

Education

Bachelor's or Master's in CS/DS/MIS or related

Tools

OpenShift

Kubernetes

LangSmith

Galileo

LangChain

Splunk

OpenAI APIs

Job description

At Bank of America, we are guided by a common purpose to help make financial lives better through the power of every connection. We do this by driving Responsible Growth and delivering for our clients, teammates, communities and shareholders every day.

Being a Great Place to Work and providing a culture of caring is core to how we drive Responsible Growth. We are intentional about fostering an inclusive workplace where every teammate has the opportunity to succeed, build a career and contribute to our shared success. This includes attracting and developing exceptional talent, recognizing and rewarding performance, and supporting our teammates' physical, emotional, and financial wellness through affordable, competitive and flexible benefits.

We value the unique perspectives individuals bring from all backgrounds and career paths - whether shaped by military service, community college education, or a wide range of work and life experiences. These journeys foster resilience, leadership and innovation, strengthening our workforce and positively impact the communities we serve.

Bank of America is committed to an in-office culture that supports collaboration, engagement, and career development. Our approach includes clear in-office expectations, while providing an appropriate level of flexibility based on role-specific responsibilities and business needs.

At Bank of America, you can build a successful career with opportunities to learn, grow, and make an impact. Join us!

Job Description

This role centers on engineering and operating third-party applications end to end within an on-premises OpenShift platform. The position supports GenAI operations, including guardrails and observability for LLM and RAG-based applications across the enterprise. The engineer will deploy and support vendor platforms such as Galileo and LangSmith, monitor AI application quality and safety, maintain model evaluation workflows, and provide operational support in a fast-paced environment.

The domain is GenAI operations: the applications you run provide guardrails and observability for LLM-and-RAG-powered systems across the enterprise. You will work with these vendor tools to monitor application quality, safety, and performance-tracking metrics such as attribution, grounding, prompt-injection detection, tone, bias, and PII handling-and surface them through dashboards and alerting. Where the platform uses model-based evaluation (LLMs-as-Judges and specialized/small language models), you will help operationalize, calibrate, and maintain those workflows.

Because the AI landscape changes quickly, the ideal engineer learns new tools and environments fast, brings a pragmatic build-and-operate mindset, and can lead cross-functional work to move initiatives forward.

Responsibilities

• Develop, deploy, and operate vendor applications in an on-premises OpenShift environment.
• Build and maintain APIs and integrations connecting vendor guardrail and observability platforms to enterprise GenAI applications.
• Monitor application performance, safety, latency, throughput, and guardrail metrics.
• Implement and maintain model-based evaluation workflows, including annotation and calibration processes.
• Utilize orchestration frameworks such as LangChain for guardrail checks, prompt management, and scoring workflows.
• Build observability dashboards and alerting solutions.
• Troubleshoot production issues and optimize performance, scalability, and resilience.
• Contribute to emerging AI evaluation and guardrail strategies for agentic workflows.
• Lead and mentor engineers while driving cross-functional initiatives and engineering best practices.

Required Qualifications

• Bachelor's or Master's degree in Computer Science, Data Science, MIS, or related field, or equivalent experience.
• 10+ years of software engineering experience building and scaling production applications.
• Strong Python development experience, including common ML and data libraries.
• Hands-on experience with OpenShift or Kubernetes in production environments.
• Experience deploying and supporting LLM or RAG-based applications.
• Understanding of AI risks, agent behavior, and governance frameworks.
• Experience integrating third-party platforms and supporting vendor applications.
• Experience with OpenAI APIs, LangChain, and GenAI development frameworks.
• Knowledge of inference performance optimization and scalability.
• Experience implementing AI guardrails, observability, and model evaluation processes.
• Experience working in Agile environments.
• Proven ability to lead technical initiatives and influence stakeholders.
• Strong analytical, problem-solving, and communication skills.

Desired Qualifications

• Experience delivering technology solutions within large, regulated enterprise environments, preferably financial services.
• Experience with annotation pipelines, feedback loops, fine-tuning, and alignment techniques.
• Familiarity with prompt lifecycle management and versioning.
• Experience with Galileo, LangSmith, Splunk, or similar observability platforms.
• Familiarity with LlamaIndex, Haystack, or other GenAI frameworks.
• Experience with evaluation and guardrails for agentic AI workflows.
• Self-directed, collaborative working style across engineering teams.

Skills

• Application Development
• Automation
• Influence
• Solution Design
• Technical Strategy Development
• Architecture
• Business Acumen
• DevOps Practices
• Result Orientation
• Solution Delivery Process
• Analytical Thinking
• Collaboration
• Data Management
• Risk Management
• Test Engineering

Shift

1st shift (United States of America)

Hours Per Week

40

Pay Transparency details

US - NJ - Jersey City - 101 Hudson St - 101 Hudson (NJ2101), US - NJ - Pennington - 1600 American Blvd - Hopewell Bldg 6 (NJ2600), US - VA - Richmond - 8011 VILLA PARK DR - VILLA PARK OPS CENTER BLDG B (VA2125)

Pay and benefits information

Pay range

$102,900.00 - $179,900.00 annualized salary, offers to be determined based on experience, education and skill set.

Discretionary incentive eligible

This role is eligible to participate in the annual discretionary plan. Employees are eligible for an annual discretionary award based on their overall individual performance results and behaviors, the performance and contributions of their line of business and/or group; and the overall success of the Company.

Benefits

This role is currently benefits eligible. We provide industry-leading benefits, access to paid time off, resources and support to our employees so they can make a genuine impact and contribute to the sustainable growth of our business and the communities we serve.