Staff AI Backend Engineer

ByteHire Limited.; New York (NY); Hybrid; USD 150,000 - 250,000; Full time

Benefits offered by this job

Generous PTO

Health insurance

Life and disability insurance

Simple IRA matching

Wellness programs

Job summary

ByteHire Limited. is seeking a Staff Backend AI Engineer to design, build, and operate the backend AI systems powering an AI-driven care orchestration platform.

You will own critical components, from low-latency inference to workflow automation, collaborating with product, data, and clinical teams to ship production-grade AI systems. Ideal candidates have 8+ years building production backends, expert knowledge in distributed systems, and hands-on experience with AI-powered services, LLM

Qualifications

• 8+ years building and operating production backend systems with AI-powered services
• Experience designing and operating complex backend systems in production
• Deep understanding of scalability, reliability, and distributed systems patterns
• Experience with async or event-driven architectures
• Experience building and shipping AI-powered systems beyond prototypes
• Strong communication and cross-functional collaboration skills

Responsibilities

• Design, build, and operate the backend AI systems powering Arbiter’s care orchestration platform
• Own architectural decisions and system ownership across staff-level scope
• Mentor engineers and raise engineering standards
• Collaborate with data scientists, product, and clinical teams to translate problems into scalable solutions
• Ensure reliability, low latency, and high availability of backend services and AI pipelines
• Lead end-to-end ownership including incident response and production operations

Skills

Python

Distributed Systems

Async/Concurrency

REST/GRPC

Data Pipelines

LLM Integration

PostgreSQL

GCP

Vertex AI

BigQuery

Education

CS/Engineering degree

Tools

Vertex AI

BigQuery

Job description

Salary:

up to $250,000 per annum + bonus

About The Client:

They are building an AI-powered care orchestration platform designed to unify fragmented healthcare systems and improve patient outcomes at scale. The platform combines agentic AI workflows with a robust backend infrastructure to automate complex clinical and operational processes.

The product is backed by significant early-stage funding and led by operators with experience scaling healthcare and technology platforms nationally. The focus is on delivering a production-grade, patient-facing AI system that operates reliably in high-stakes environments.

They are looking for staff-level engineers to help design, build, and scale the backend systems that power the platform.

They operate as a high-performing engineering team focused on delivering high-quality, impactful systems. Their values include:

Engineering Excellence: Strong ownership of technical quality, performance, and reliability

Impact-Driven Development: Building systems that solve real-world problems at scale

Continuous Growth: Learning, mentorship, and raising the bar across the team

Responsibilities

As a Staff Backend AI Engineer, you will design, build, and operate the backend AI systems that power Arbiter’s intelligent care orchestration platform. You will own critical parts of our agentic infrastructure, from low-latency inference pipelines to workflow automation and service orchestration, working closely with product, data, and clinical partners to ship production-grade AI systems with real customer dependency.

Build the Agentic Platform: Design, develop, and operate backend systems powering agent-driven workflows, including orchestration layers, inference pipelines, and automation primitives

Production AI Systems: Ship and maintain production-grade AI systems automating complex clinical and administrative workflows with strict reliability and latency requirements

Platform Architecture & Scalability: Contribute to architecture with a focus on modularity, extensibility, and integration with EMRs, data systems, and external services

Inference & Backend Reliability: Ensure high performance, low latency, and fault tolerance across backend services and AI pipelines

Infrastructure & Deployment: Implement best practices across CI/CD, testing, observability, and production deployments

Cross-Functional Execution: Partner with data scientists, product, and clinical teams to translate complex problems into scalable systems

Technical Leadership & Mentorship: Raise the engineering bar through mentorship, standards, and technical direction

Ownership & On-Call: Own systems end-to-end, including incident response and operational excellence

Skills & Experience

Strong experience building complex production systems in Python, with a focus on scalability and reliability

Proven ability to design and operate highly scalable, distributed systems in production

Deep expertise in event-driven architectures, concurrency (multi-processing, multi-threading, async), and distributed system design

Strong experience designing and building distributed services (REST and/or gRPC) and service-oriented architectures

Experience designing data processing pipelines, including intelligent decision‑making systems and workflow orchestration

Hands-on experience building AI-powered systems, including LLM integrations and understanding trade-offs (latency, cost, accuracy)

Experience implementing agentic workflows or finite state machines for complex, multi-step processes

Strong experience with metrics, observability, and evaluation systems to measure reliability, performance, and scalability

Experience working with large-scale data processing and analytics (e.g. pandas, BigQuery or similar)

Solid experience with PostgreSQL and data modelling at scale (e.g. multi-tenant systems, performance optimisation)

Production experience with cloud infrastructure (GCP preferred), including AI and data platforms (e.g. Vertex AI, BigQuery or similar)

Strong practices across CI/CD, monitoring, and production operations

Experience integrating with external systems/APIs, ideally in complex domains (e.g. healthcare, EHR/EMR)

Demonstrated ability to operate at staff level, including system ownership, architectural decision-making, and mentoring engineers

Minimum Qualifications

8+ years of hands-on experience building and operating production backend systems, including recent experience with AI-powered services

Computer Science or Engineering degree, or equivalent practical experience

Proven experience designing and operating complex backend systems in production

Deep understanding of scalability, reliability, and distributed systems patterns

Experience with async or event-driven architectures

Experience building and shipping AI-powered systems beyond prototypes

Strong communication and cross-functional collaboration skills

Preferred Qualifications

Experience building agentic workflows, including multi-agent systems and human-in-the-loop patterns

Experience integrating LLM APIs, prompt engineering, and evaluation techniques

Familiarity with AI-assisted development tools (e.g. Cursor, Claude Code, GitHub Copilot)

Track record of delivering complex systems end-to-end with minimal oversight

Experience in startup or high-growth environments

Location

This role can be remote or onsite, based in New York City or Boca Raton, in a fast-paced, collaborative environment where ideas move quickly from concept to production.

We offer a comprehensive and competitive benefits package designed to support your well‑being and professional growth:

Generous Paid Time Off (PTO) to support work-life balance

Comprehensive health, vision, and dental insurance for you and your family

Life and disability insurance for financial security

Simple IRA matching to support long-term financial goals

Wellness programs supporting physical and mental health