GenAI Software Engineer — Healthcare LLM Systems

Abridge; New York (NY); Hybrid; USD 120,000 - 190,000; Full time

Benefits offered by this job

Generous Time Off

Comprehensive Health Plans

401(k) Matching

Paid Parental Leave

Mental Health Support

Sabbatical Leave

Equity grants

Lifestyle Wallet

Job summary

Join Abridge as a GenAI Software Engineer and help design agentic LLM workflows that transform clinical conversations into reliable, real-time outputs. You will work with researchers, clinicians, and engineers to build end-to-end systems with rigorous evaluation and deployment in low-latency environments.

You’ll prototype with new models, manage tool use and retrieval, and contribute to scalable, observable production pipelines while collaborating across SF/NYC offices.

Qualifications

• 3+ years of experience building production-grade systems, with 1-2+ years on LLM-powered or agentic products.
• Deep fluency with LLM APIs, prompting strategies, and orchestration patterns (LangChain, LlamaIndex, custom pipelines).
• Experience with retrieval systems (semantic and lexical), vector DBs, efficient kNN, function calling, tool-use, or agentic workflows.
• Working knowledge of model evaluation, building diverse datasets, automated and human-in-the-loop evaluations, and A/B testing.
• Strong Python fundamentals, including clean code, test cases, async programming, performance profiling, packaging, and deployment tooling.
• Good taste and judgment on when to move fast vs. address tech debt.

Responsibilities

• Design and build agentic systems that turn LLMs into dependable tools with retrieval, tool use, and structured outputs.
• Collaborate with ML and infra engineers to scale and optimize agentic workflows, managing latency and context windows.
• Write high-quality, modular code that remains robust under failure and easy to iterate on.
• Own major architectural decisions—data flow, state caching, and structuring generative outputs.
• Drive rigorous evaluation with benchmark datasets and human-in-the-loop frameworks; run experiments to surface edge cases.
• Leverage frontier capabilities to prototype with new models and open-source tools.

Skills

LLM APIs

Prompting strategies

Orchestration patterns

LangChain

LlamaIndex

Custom pipelines

Retrieval systems

Vector DBs

Function calling

Tool-use

Agentic workflows

Python fundamentals

Async programming

Testing & deployment tooling

Tools

LangChain

LlamaIndex

Vector DBs

kNN

Integration tooling

Job description

Join Abridge as a GenAI Software Engineer and help design agentic LLM workflows that transform clinical conversations into reliable, real-time outputs. You will work with researchers, clinicians, and engineers to build end-to-end systems with rigorous evaluation and deployment in low-latency environments.

You’ll prototype with new models, manage tool use and retrieval, and contribute to scalable, observable production pipelines while collaborating across SF/NYC offices.