Senior GenAI Systems Engineer - Production RAG & MLOps

BNY; Jersey City (NJ); On-site; USD 120,000 - 200,000; Full time

Job summary

BNY is seeking a senior-level engineer to design, build, and operate production-grade GenAI and RAG platforms at scale in Jersey City, NJ. The role focuses on industrializing LLM-based systems with guardrails, observability, evaluation frameworks, and operational rigor.

You will own end-to-end lifecycle management, contribute to safety, cost efficiency, and collaborate with governance to meet enterprise standards; mentoring engineers and shaping best practices within a reliability-focused team.

Qualifications

• Strong experience building and operating production ML or GenAI systems in enterprise environments.
• Hands-on expertise with LLM orchestration frameworks (LangChain and/or LlamaIndex).
• Experience with model registries and experiment tracking (MLflow or equivalent).
• Solid understanding of Kubernetes-based deployments and cloud-native architectures.
• Familiarity with feature stores, data pipelines, and retriever/index lifecycle management.
• Proven telemetry, logging, metrics, and distributed tracing for ML/AI workloads.
• Strong CI/CD practices for ML, GenAI, and data-driven systems.

Responsibilities

• Design and build production-ready RAG pipelines with guardrails and observability.
• Develop offline/online evaluation frameworks for prompts, models, datasets.
• Own end-to-end lifecycle management for GenAI systems.
• Establish CI/CD pipelines for prompts, models, and data.
• Implement cost and performance monitoring and spend optimization.
• Build safety mechanisms including content filtering and abuse detection.
• Define incident management workflows with alerting and rollback.
• Collaborate with product, platform, and governance teams for compliance.
• Mentor engineers and propagate best practices for scalable AI.

Skills

Production ML/GenAI

LangChain

LlamaIndex

MLflow

Kubernetes

Feature stores

Data pipelines

Retriever/index lifecycle

Telemetry/ observability

CI/CD for ML

Tools

MLflow

Kubernetes

LangChain

LlamaIndex

Job description

BNY is seeking a senior-level engineer to design, build, and operate production-grade GenAI and RAG platforms at scale in Jersey City, NJ. The role focuses on industrializing LLM-based systems with guardrails, observability, evaluation frameworks, and operational rigor.

You will own end-to-end lifecycle management, contribute to safety, cost efficiency, and collaborate with governance to meet enterprise standards; mentoring engineers and shaping best practices within a reliability-focused team.