Staff ML Engineer: AI Platform Orchestration

Freshworks; San Mateo (CA); Hybrid; USD 211,000 - 261,000; Full time

Benefits offered by this job

Dental insurance

Medical insurance

Vision insurance

Equity + ESPP

Flexible PTO

Commuter benefits

Wellness benefits

Parental leave

Job summary

Freshworks in San Mateo, CA is seeking a Senior Machine Learning Staff Engineer to lead backend services powering the Agentic AI Platform. You will own agent orchestration, API design, and scalable runtimes, collaborating with the Principal AI Architect to bridge design and implementation.

You will build high-performance APIs, integrate LangChain, LangGraph, and LangSmith, and mentor junior engineers in a fast-paced SaaS environment while upholding engineering excellence and security.

Qualifications

• 6+ years of backend engineering experience.
• 1-2+ years hands-on experience in AI/ML platform development.
• Expert proficiency in Java and Python.
• Framework expertise with LangChain, LangGraph, or LangSmith.
• Distributed systems knowledge: event-driven architectures, microservices, and Kubernetes.
• Cloud & Data experience with AWS, Terraform, relational PostgreSQL, and vector databases.
• Orchestration experience with workflow engines like Temporal or Airflow and Kafka.

Responsibilities

• Platform Backend Development: lead agent runtime orchestration services, focusing on reasoning, planning, and tool invocation.
• API & SDK Design: own robust APIs and SDKs for internal/external teams to build and deploy agent workflows.
• State & Memory Management: build stateful dialog management and memory services for multi-turn agents.
• Agent Communication: implement multi-agent communication protocols and shared context systems.
• Microservices Leadership: transition to cloud-native backend services with microservices or service mesh architectures.
• Workflow Design: design task scheduling and long-running operations using Temporal or Airflow.
• Performance Optimization: optimize LLM orchestration for cost and performance via caching and batching.
• Security & Reliability: implement RBAC, API authentication, and OpenTelemetry for observability.

Skills

Backend engineering

AI/ML platform development

Java

Python

Distributed systems

Kubernetes

PostgreSQL

Vector databases

REST APIs

Kafka

Tools

LangChain

LangGraph

LangSmith

Temporal

Airflow

Kafka

Terraform

AWS

PostgreSQL

Vector databases

Job description

Freshworks in San Mateo, CA is seeking a Senior Machine Learning Staff Engineer to lead backend services powering the Agentic AI Platform. You will own agent orchestration, API design, and scalable runtimes, collaborating with the Principal AI Architect to bridge design and implementation.

You will build high-performance APIs, integrate LangChain, LangGraph, and LangSmith, and mentor junior engineers in a fast-paced SaaS environment while upholding engineering excellence and security.