Staff Backend AI Platform Engineer

Freshworks; California (MO); Hybrid; USD 225,000 - 278,000; Full time

Benefits offered by this job

Dental, medical, and vision insurance

Flexible PTO

Commuter benefits

Wellness benefits

Adoption and parental leave benefits

Job summary

Freshworks is seeking a Backend AI Platform Engineer to lead the development of backend services for our Agentic AI Platform. This role will involve building reasoning-driven agents and high-performance APIs, collaborating closely with the Principal AI Architect.

The successful candidate will have at least 6 years of backend engineering experience, with expertise in Java and Python. Our hybrid model requires presence 3 times a week from our San Mateo, CA headquarters. The compensation range is between $225,000 – $278,000 annually, plus a target bonus.

Qualifications

• 6+ years of backend engineering experience.
• 1-2+ years of hands-on experience in AI/ML platform development.
• Expert proficiency in Java and Python.
• Hands-on experience with LangChain, LangGraph, or LangSmith.
• Deep understanding of event-driven architectures, microservices, and Kubernetes.
• Solid experience with AWS and relational as well as vector databases.
• Practical experience with workflow engines and messaging systems.

Responsibilities

• Lead engineer for agent runtime orchestration services.
• Own the development of robust APIs and SDKs.
• Build and optimize stateful dialog management.
• Implement multi-agent communication protocols.
• Execute the transition to cloud-native backend services.
• Build and manage task scheduling and long-running operations.
• Optimize LLM orchestration for performance and cost.
• Implement and maintain data pipelines.
• Ensure backend services support evaluation and observability.
• Implement RBAC and API authentication.

Skills

Backend engineering experience

AI/ML platform development

Java proficiency

Python proficiency

LangChain experience

Event-driven architectures

Microservices knowledge

Kubernetes knowledge

AWS knowledge

Experience with PostgreSQL

Tools

Terraform

Temporal

Airflow

Kafka

Job description

We are seeking a Backend AI Platform Engineer to lead the development of core backend services powering our Agentic AI Platform. In this role, you will be the primary technical driver for building reasoning-driven agents, multi‑agent orchestration, and outcome-based workflows. You will collaborate with the Principal AI Architect to bridge the gap between high‑level design and scalable implementation. You will own the development of high-performance APIs and orchestration runtimes, integrating frameworks like LangChain, LangGraph, and LangSmith. This role is highly hands‑on and requires a technical leader who can navigate a fast‑paced environment while maintaining high engineering standards.

Key Responsibilities

Platform Backend Development

• Lead Implementation: Act as the lead engineer for agent runtime orchestration services, focusing on reasoning, planning, and tool invocation.
• API & SDK Design: Own the development of robust APIs and SDKs that enable internal and external teams to build and deploy agent workflows.
• State & Memory Management: Build and optimize stateful dialog management and memory services for multi‑turn, context-aware agents.
• Agent Communication: Implement multi-agent communication (A2A) protocols and shared context systems.

System Architecture & Execution

• Microservices Leadership: Execute the transition to cloud‑native, event-driven backend services using microservices or service mesh architectures.
• Workflow Design: Build and manage task scheduling and long-running operations using tools like Temporal or Airflow.
• Performance Optimization: Optimize LLM orchestration for performance and cost through caching, batching, and token monitoring.

Integrations & Data Services

• Data Pipeline Ownership: Implement and maintain RAG pipelines and integrations with vector databases (e.g., Pinecone, Weaviate, FAISS).
• Telemetry Integration: Ensure all backend services support LangSmith-based evaluation and comprehensive observability.

Security & Reliability

• Tenant Isolation: Implement RBAC, API authentication, and multi-tenant isolation logic.
• Engineering Excellence: Set the standard for automated testing (unit, integration, load) and implement OpenTelemetry for full system visibility.
• Mentorship: Provide technical guidance and code reviews for more junior backend engineers on the team.

Required Qualifications

• 6+ years of backend engineering experience.
• 1-2+ years of hands‑on experience in AI/ML platform development.
• Expert proficiency in Java and Python.
• Hands‑on experience with LangChain, LangGraph, or LangSmith.
• Deep understanding of event-driven architectures, microservices, and Kubernetes.
• Solid experience with AWS, Terraform, and relational (PostgreSQL) as well as vector databases.
• Practical experience with workflow engines like Temporal or Airflow and messaging systems like Kafka.

Preferred Qualifications

• Experience with AI-driven planning systems or dialog state tracking.
• Background in enterprise security (RBAC, SSO, OAuth2).
• Experience working in a high-growth SaaS or startup environment.
• Active contributions to open-source backend or AI frameworks.

Hybrid role – onsite expectations of 3x/week (Tues – Thurs) from our San Mateo, CA headquarters.

The annual base salary range for this position is $225,000 – $278,000. This role is also eligible for a target bonus.

Compensation is based on a variety of factors, including but not limited to location, experience, job-related skills, and level.

Benefits

Freshworks offers multiple options for dental, medical, vision, disability, and life insurance. Equity + ESPP, flexible PTO, flexible spending, commuter benefits, and wellness benefits are also offered. Freshworks also offers adoption and parental leave benefits.

We are committed to providing equal opportunity and believe that diversity in the workplace creates a more vibrant, richer environment that boosts the goals of our employees, communities, and business.