Senior AI Engineer – LLM-Driven Healthcare Platforms

Ro; New York (NY); On-site; USD 182,000 - 220,000; Full time

Job summary

Ro is building a patient-centric tech platform in New York, delivering AI-powered healthcare experiences. We seek a Senior AI Engineer to own features from prototype to production, focusing on orchestration layers, retrieval, embeddings, and safe, reliable operations that scale with clinical workflows.

You will collaborate with EHR operators, clinicians, and product teams to ship impactful AI capabilities, measure outcomes, and iterate rapidly in a high-visibility environment.

Qualifications

• 5+ years building production software, with 1–2+ years deploying LLM-powered apps.
• Strong Python engineer with backend services experience and AI API/framework integration.
• Experience shipping AI features with measurable quality, reliability, latency, and cost impact.

Responsibilities

• Design, build, and ship production LLM-powered features end-to-end.
• Build AI application layer: prompt orchestration, tool calling, retrieval (RAG), embeddings, agent workflows, and outputs.
• Develop evaluation infrastructure: regression suites, LLM-as-a-judge, synthetic data, human review workflows, quality metrics.
• Design observability to measure accuracy, latency, cost, and model behavior in production.
• Build safety and reliability guardrails for quality, compliance, and operations.
• Evaluate and integrate third-party tools and make build-vs-buy decisions.
• Raise engineering standards through leadership, code reviews, and mentorship.
• Collaborate with clinicians and operators to understand workflows to automate.

Skills

Production software

Python

AI APIs

Product impact

Product sense

Orchestration frameworks

RAG

Vector databases

Prompt management

Observability

Tools

LangChain

LangGraph

AgentCore

Vector databases

LangSmith

Arize Phoenix

Braintrust

Job description

Ro is building a patient-centric tech platform in New York, delivering AI-powered healthcare experiences. We seek a Senior AI Engineer to own features from prototype to production, focusing on orchestration layers, retrieval, embeddings, and safe, reliable operations that scale with clinical workflows.

You will collaborate with EHR operators, clinicians, and product teams to ship impactful AI capabilities, measure outcomes, and iterate rapidly in a high-visibility environment.