Staff Backend AI Platform Engineer

AItoolnavio

San Mateo (CA)

Hybrid

USD 225,000 - 278,000

Full time

3 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Dental insurance
Medical insurance
Vision insurance
Disability insurance
Life insurance
Equity + ESPP
Flexible PTO
Commuter benefits
Wellness benefits
Adoption leave
Parental leave

Job summary

Freshworks seeks a Backend AI Platform Engineer to lead core backend services powering the Agentic AI Platform. You will drive reasoning-driven agents, multi-agent orchestration, and outcome-based workflows, collaborating with the Principal AI Architect to bridge design and scalable implementation.

The role requires hands-on leadership, API/SDK design, and expertise with LangChain, LangGraph, LangSmith, and cloud-native microservices. Hybrid onsite 3x/week at our San Mateo, CA headquarters.

Qualifications

  • 6+ years of backend engineering experience.
  • 1-2+ years of hands-on experience in AI/ML platform development.
  • Expert proficiency in Java and Python.
  • Hands-on experience with LangChain, LangGraph, or LangSmith.
  • Distributed systems knowledge: event-driven architectures, microservices, Kubernetes.

Responsibilities

  • Lead implementation of agent runtime orchestration services focusing on reasoning, planning, and tool invocation.
  • Own development of robust APIs and SDKs for internal and external teams.
  • Build and optimize stateful dialog management and memory services for multi-turn agents.
  • Implement multi-agent communication (A2A) and shared context systems.
  • Transition to cloud-native backend services using microservices or service mesh architectures.

Skills

Backend engineering
Java
Python
Distributed systems
Event-driven architectures
Kubernetes

Tools

LangChain
LangGraph
LangSmith
AWS
Terraform
PostgreSQL
Pinecone
Weaviate
FAISS
Temporal
Airflow
Kafka

Job description

Job Description

We are seeking a Backend AI Platform Engineer to lead the development of core backend services powering our Agentic AI Platform. In this role, you will be the primary technical driver for building reasoning-driven agents, multi-agent orchestration, and outcome-based workflows.

You will collaborate with the Principal AI Architect to bridge the gap between high-level design and scalable implementation. You will own the development of high-performance APIs and orchestration runtimes, integrating frameworks like LangChain, LangGraph, and LangSmith. This role is highly hands-on and requires a technical leader who can navigate a fast-paced environment while maintaining high engineering standards.

Key Responsibilities

Platform Backend Development

  • Lead Implementation: Act as the lead engineer for agent runtime orchestration services, focusing on reasoning, planning, and tool invocation.
  • API & SDK Design: Own the development of robust APIs and SDKs that enable internal and external teams to build and deploy agent workflows.
  • State & Memory Management: Build and optimize stateful dialog management and memory services for multi-turn, context-aware agents.
  • Agent Communication: Implement multi-agent communication (A2A) protocols and shared context systems.

System Architecture & Execution

  • Microservices Leadership: Execute the transition to cloud-native, event-driven backend services using microservices or service mesh architectures.
  • Workflow Design: Build and manage task scheduling and long-running operations using tools like Temporal or Airflow.
  • Performance Optimization: Optimize LLM orchestration for performance and cost through caching, batching, and token monitoring.

Integrations & Data Services

  • Data Pipeline Ownership: Implement and maintain RAG pipelines and integrations with vector databases (e.g., Pinecone, Weaviate, FAISS).
  • Telemetry Integration: Ensure all backend services support LangSmith-based evaluation and comprehensive observability.

Security & Reliability

  • Tenant Isolation: Implement RBAC, API authentication, and multi-tenant isolation logic.
  • Engineering Excellence: Set the standard for automated testing (unit, integration, load) and implement OpenTelemetry for full system visibility.
  • Mentorship: Provide technical guidance and code reviews for more junior backend engineers on the team.
Qualifications

Required Qualifications:

  • 6+ years of backend engineering experience.
  • 1-2+ years of hands-on experience in AI/ML platform development.
  • Expert Proficiency: Strong programming skills in Java and Python.
  • Framework Expertise: Hands-on experience with LangChain, LangGraph, or LangSmith.
  • Distributed Systems: Deep understanding of event-driven architectures, microservices, and Kubernetes.
  • Cloud & Data: Solid experience with AWS, Terraform, and a mix of relational (PostgreSQL) and Vector databases.
  • Orchestration: Practical experience with workflow engines like Temporal or Airflow and messaging systems like Kafka.

Preferred Qualifications:

  • Experience with AI-driven planning systems or dialog state tracking.
  • Background in enterprise security (RBAC, SSO, OAuth2).
  • Experience working in a high-growth SaaS or startup environment.
  • Active contributions to open-source backend or AI frameworks.
Additional Information

Please note this is a hybrid role with onsite expectations of 3x/week (Tues - Thurs) from our San Mateo, CA headquarters.

The annual base salary range for this position is $225,000- $278,000.This role is also eligible for a target bonus.

Compensation is based on a variety of factors, including but not limited to location, experience, job-related skills, and level.

Freshworks offers multiple options for dental, medical, vision, disability, and life insurance. Equity + ESPP, flexible PTO, flexible spending, commuter benefits, and wellness benefits are also offered. Freshworks also offers adoption and parental leave benefits.

At Freshworks, we have fostered an environment that enables everyone to find their true potential, purpose, and passion, welcoming colleagues of all backgrounds, genders, sexual orientations, religions, and ethnicities. We are committed to providing equal opportunity and believe that diversity in the workplace creates a more vibrant, richer environment that boosts the goals of our employees, communities, and business. Fresh vision. Real impact. Come build it with us.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Staff Engineer – Machine Learning
Staff Engineer – Machine Learning

AItoolnavio • San Mateo (CA)

Hybrid
USD 211,000 - 261,000
Dental insurance
Medical insurance
Vision insurance
+8
Staff Backend AI Platform Engineer
Staff Backend AI Platform Engineer

Aimlroles • San Mateo (CA)

Hybrid
USD 225,000 - 278,000
Hybrid on-site 3x/week (San Mateo)
Equity + ESPP
Flexible PTO
+3
Staff Engineer - Machine Learning
Staff Engineer - Machine Learning

Freshworks • San Mateo (CA)

Hybrid
USD 211,000 - 261,000
Dental insurance
Medical insurance
Vision insurance
+5
Senior Director, AI Platform (Product Management)
Senior Director, AI Platform (Product Management)

AItoolnavio • San Mateo (CA)

Hybrid
USD 290,000 - 360,000
Variable compensation
Bonus/equity available
Senior Backend AI Platform Engineer
Senior Backend AI Platform Engineer

Aimlroles • San Mateo (CA)

Hybrid
USD 225,000 - 278,000
Hybrid on-site 3x/week (San Mateo)
Equity + ESPP
Flexible PTO
+3
Backend Engineer
Backend Engineer

Goliath Partners Inc. • San Francisco (CA)

Hybrid
USD 270,000 - 315,000
Comprehensive health coverage
401(k) retirement plan
Visa sponsorship available
+1
Principal Solutions Engineer
Principal Solutions Engineer

Freshworks • San Mateo (CA)

On-site
USD 163,000 - 234,000
Dental, medical, vision insurance
Equity + ESPP
Flexible PTO
+2
Staff ML Engineer: AI Platform Orchestration
Staff ML Engineer: AI Platform Orchestration

Freshworks • San Mateo (CA)

Hybrid
USD 211,000 - 261,000
Dental insurance
Medical insurance
Vision insurance
+5
Senior Backend Engineer
Senior Backend Engineer

Goliath Partners Inc. • San Francisco (CA)

Hybrid
USD 280,000 - 325,000
Medical benefits
401(k) plan
Visa sponsorship
Staff Software Engineer
Staff Software Engineer

Cantor Fitzgerald • Chicago (IL)

On-site
USD 190,000 - 250,000