Production AI/ML Engineer - GenAI & LLM Ops (NY On-Site)

OP Recruiting

New York (NY)

On-site

USD 250,000 - 350,000

Full time

4 days ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Benefits offered by this job

Health insurance
Dental insurance
Vision insurance
401(k) with company match

Job summary

OP Recruiting in New York, NY is seeking an Applied AI / Machine Learning Engineer to build the AI platform, agentic workflows, and high-throughput retrieval systems in a fast-paced financial services environment.

You will design production-grade LLM infrastructure, optimize performance, and deploy models across distributed cloud environments with a strong emphasis on reliability and impact.

Qualifications

  • 6+ years of software engineering and ML experience (or 4+ with advanced quantitative degree).
  • Experience building production applications with LLM orchestration tools and protocol-driven integration.
  • Experience deploying on distributed cloud environments (AWS, Azure, GCP).

Responsibilities

  • Design and maintain high-efficiency AI infrastructure with low-latency inference on dedicated hardware.
  • Develop autonomous agentic workflows for complex data tasks and software development.
  • Scale retrieval architectures using RAG models, vector stores, and embedding pipelines.
  • Collaborate with cross-functional teams to deliver AI-driven production applications.
  • Establish robust MLOps practices including CI/CD, monitoring, drift management, and quality metrics.

Skills

Python programming
PyTorch
JAX
LLM orchestration
Vector storage
Containers
Distributed cloud
MLOps
Low-latency inference

Tools

Docker

Job description

OP Recruiting in New York, NY is seeking an Applied AI / Machine Learning Engineer to build the AI platform, agentic workflows, and high-throughput retrieval systems in a fast-paced financial services environment.

You will design production-grade LLM infrastructure, optimize performance, and deploy models across distributed cloud environments with a strong emphasis on reliability and impact.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

AI/ML Engineer — LLM Infra & Agentic Workflows
AI/ML Engineer — LLM Infra & Agentic Workflows

Objective Partners • New York (NY)

On-site
USD 150,000 - 260,000
Medical, dental, vision insurance
High-performance hardware access
Competitive base + incentive options
Senior Machine Learning Engineer – Production AI & MLOps
Senior Machine Learning Engineer – Production AI & MLOps

So • New York (NY)

On-site
USD 140,000 - 190,000
Applied AI/ML Engineer
Applied AI/ML Engineer

OP Recruiting • New York (NY)

On-site
USD 250,000 - 350,000
Health insurance
Dental insurance
Vision insurance
+1
Production AI Engineer - Real-World ML for Business (Hybrid NYC)
Production AI Engineer - Real-World ML for Business (Hybrid NYC)

People in AI • New York (NY)

Hybrid
USD 120,000 - 180,000
Hybrid NYC AI Engineer: Build LLM Agents for Finance
Hybrid NYC AI Engineer: Build LLM Agents for Finance

Motive Partners • New York (NY)

Hybrid
USD 170,000 - 300,000
Applied AI Engineer — Production ML & LLMs (Hybrid NYC)
Applied AI Engineer — Production ML & LLMs (Hybrid NYC)

Logicbroker • New York (NY)

Hybrid
USD 102,000 - 138,000
Applied AI/ML Engineer
Applied AI/ML Engineer

Objective Partners • New York (NY)

On-site
USD 150,000 - 260,000
Medical, dental, vision insurance
High-performance hardware access
Competitive base + incentive options
Applied AI Engineer: Agentic AI & LLMs – Onsite NYC
Applied AI Engineer: Agentic AI & LLMs – Onsite NYC

Robson Bale Ltd • New York (NY)

On-site
USD 150,000 - 200,000
Competitive base salary
Medical, dental, vision coverage
401(k) with matching
+4
Senior AI Engineer - GenAI & LLM Orchestration
Senior AI Engineer - GenAI & LLM Orchestration

LTM • New York (NY)

On-site
USD 130,000 - 190,000
Comprehensive Medical Plan
Short Term and Long-Term Disability
401(k) Plan with Company match
+2
Finance AI Engineer: LLMs, RAG & Production Apps
Finance AI Engineer: LLMs, RAG & Production Apps

Social Capital Resources • New York (NY)

On-site
USD 130,000 - 190,000