Senior AI Engineer

Ro; New York (NY); On-site; USD 182,000 - 220,000; Full time

Job summary

Ro is building a patient-centric tech platform in New York, delivering AI-powered healthcare experiences. We seek a Senior AI Engineer to own features from prototype to production, focusing on orchestration layers, retrieval, embeddings, and safe, reliable operations that scale with clinical workflows.

You will collaborate with EHR operators, clinicians, and product teams to ship impactful AI capabilities, measure outcomes, and iterate rapidly in a high-visibility environment.

Qualifications

• 5+ years building production software, with 1–2+ years deploying LLM-powered apps.
• Strong Python engineer with backend services experience and AI API/framework integration.
• Experience shipping AI features with measurable quality, reliability, latency, and cost impact.

Responsibilities

• Design, build, and ship production LLM-powered features end-to-end.
• Build AI application layer: prompt orchestration, tool calling, retrieval (RAG), embeddings, agent workflows, and outputs.
• Develop evaluation infrastructure: regression suites, LLM-as-a-judge, synthetic data, human review workflows, quality metrics.
• Design observability to measure accuracy, latency, cost, and model behavior in production.
• Build safety and reliability guardrails for quality, compliance, and operations.
• Evaluate and integrate third-party tools and make build-vs-buy decisions.
• Raise engineering standards through leadership, code reviews, and mentorship.
• Collaborate with clinicians and operators to understand workflows to automate.

Skills

Production software

Python

AI APIs

Product impact

Product sense

Orchestration frameworks

RAG

Vector databases

Prompt management

Observability

Tools

LangChain

LangGraph

AgentCore

Vector databases

LangSmith

Arize Phoenix

Braintrust

Job description

Join Tech @ Ro to build the future of healthcare, from the ground up!

At Ro, we believe that when people achieve their health goals, they can achieve their life goals. The highest-leverage way to move society forward is to give people their health, and the current healthcare system isn’t built to do that. It was built to bill, not to serve patients.

We’re building a new system. One where the patient is in control. One designed from scratch for the digital age.

At Ro, technology isn’t just a function… It’s core to how we deliver care.

…and we’re just getting started.

As part of Tech @ Ro, you’ll work on systems that operate at scale, with an opportunity to:

• Solve complex, high-concurrency problems across a full-stack platform
• Build and ship quickly with tight feedback loops and real-world impact
• Own systems end-to-end, from architecture to production performance
• Work alongside experienced operators, technical leaders, and clinicians
• Help define how modern healthcare should be delivered

We’re a performance-driven team with a strong sense of ownership and urgency. We move fast, learn quickly, and hold a high bar for what we build, and do so with a big heart — because patients depend on it.

If you’re motivated by impact, scale, and the chance to help lead the patient revolution, come build with us.

The Role

Ro is building a team focused on shipping LLM-powered products across the patient experience, clinical operations, and internal tooling.

We’re hiring a Senior AI Engineer to join a new team we’re building from the ground up alongside a Senior AI Lead and an Engineering Manager. You’ll own AI features from prototype through production and spend as much time improving workflows with clinical operators as writing code. Most of your work will involve building orchestration layers, prompts, evaluations, retrieval systems, and tooling that make LLMs reliable enough for real-world healthcare operations.

What You’ll Do

• Design, build, and ship production LLM powered features end-to-end.
• Build the AI application layer, including prompt orchestration, tool calling, retrieval (RAG), embeddings, agent workflows, and structured outputs.
• Develop evaluation infrastructure, including regression suites, LLM-as-a-judge evaluations, synthetic datasets, human review workflows, and quality metrics that catch regressions before users do.
• Design observability systems that measure accuracy, latency, cost, hallucination rates, and model behavior in production.
• Build safety and reliability guardrails that ensure AI systems meet quality, compliance, and operational requirements.
• Evaluate and integrate third-party tools, including orchestration frameworks, evaluation platforms, vector databases, and model providers; and make pragmatic build-versus-buy decisions.
• Raise engineering standards across the team through technical leadership, code reviews, mentorship, and thoughtful system design.
• Sit with EHR operators, Patient Advocates, and clinical teams to understand the workflows you’re automating.
• Prototype quickly, ship early, measure outcomes, and iterate based on real-world usage.

What You’ll Bring to the Team

• 5+ years building production software, with at least 1–2 years building and deploying LLM-powered applications in production (or equivalent depth through substantial side projects or open-source work).
• Strong Python engineer with experience building backend services and integrating modern AI APIs and frameworks.
• Comfortable using AI-assisted development tools throughout the software development lifecycle.
• You’ve shipped AI features that served real users, and you can clearly articulate how you measured quality, reliability, latency, cost, and business impact.
• You have strong product instincts and thrive in ambiguous environments, translating operational problems into technical solutions without relying on detailed specifications.
• Bonus: Experience with orchestration frameworks (LangChain, LangGraph, AgentCore), RAG architectures, vector databases, prompt/version management, and evaluation or observability platforms (e.g., LangSmith, Braintrust, Arize Phoenix).

A note on reporting structure

This is a new function at Ro, and we’re being deliberate about not over-defining it. Your manager and where you sit on the org chart will depend on the specific shape of the team we end up with. We’d rather find the right people and figure out the lines around them than pre-draw boxes and miss great candidates. If that ambiguity is a deal-breaker, this isn’t the right role; if it sounds like an opportunity, we want to talk.

The target base salary for this position ranges from $182,300 to $220,000, in addition to a competitive equity and benefits package (as applicable). When determining compensation, we analyze and carefully consider several factors, including location, job-related knowledge, skills and experience. These considerations may cause your compensation to vary.

Ro is consistently recognized as a top workplace in Health Care, in New York, and for Women and Parents—earning more than 20 honors from Fortune, Great Place to Work, and PEOPLE since 2021. In 2025 alone, we ranked top 5 among medium workplaces in Health Care and New York, and top 50 nationwide.

At Ro, we believe that our diverse perspectives are our biggest strengths — and that embracing them will create real change in healthcare. As an equal opportunity employer, we provide equal opportunity in all aspects of employment, including recruiting, hiring, compensation, training and promotion, termination, and any other terms and conditions of employment without regard to race, ethnicity, color, religion, sex, sexual orientation, gender identity, gender expression, familial status, age, disability and/or any other legally protected classification protected by federal, state, or local law.

Ro is committed to providing reasonable accommodations for qualified individuals with disabilities in our application and interview process. If you require a reasonable accommodation in the application or interview process, please contact us at [e-mail].

See our California Privacy Policy here.