Distributed Systems Engineer: AI Agent Orchestration

ServiceNow; Mountain View (CA); On-site; USD 180,000 - 260,000; Full time

Job summary

ServiceNow in Mountain View, CA is hiring a distributed systems engineer to build the runtime infrastructure powering Moveworks' AI agents, emphasizing orchestration, execution, and real-time delivery of agent responses. This is not an individual ML engineering role.

You will own an agent orchestration engine, implement lease-based session management, and optimize an event-driven pipeline using SQS, Kafka, gRPC, and Socket.IO, with Python and Go for production-grade concurrency.

Qualifications

• Designing distributed systems with strong consistency and idempotency.
• Building asynchronous, event-driven services with robust error handling.
• Experience with message queues (SQS, Kafka) and pub/sub patterns.
• Proficiency with DynamoDB and Redis for state management.
• Skills in observability using OpenTelemetry and metrics collection.
• Experience with gRPC and protobuf-based services.
• 2+ years building production backend or infrastructure systems.
• Proficiency in Python and/or Go, with preference for both.

Responsibilities

• Develop and maintain an agent orchestration engine coordinating long-running sessions across multiple LLM calls.
• Implement lease-based ownership, heartbeats, and crash recovery via checkpoints.
• Operate and optimize SQS FIFO queues, Kafka consumers, and real-time streaming via gRPC/Socket.IO.
• Leverage Python asyncio TaskGroups for concurrent tasks with fail-fast semantics.
• Instrument systems with OpenTelemetry; propagate trace context across async boundaries.
• Manage Redis and DynamoDB state stores with per-org/per-bot scoping.

Skills

Distributed systems

Async programming

Event-driven architectures

Databases/Storage

Observability

gRPC/protobuf

Backend development

Python/Go

Tools

DynamoDB

Redis

SQS

Kafka

Socket.IO

OpenTelemetry

Prometheus

protobuf

gRPC

Job description

ServiceNow in Mountain View, CA is hiring a distributed systems engineer to build the runtime infrastructure powering Moveworks' AI agents, emphasizing orchestration, execution, and real-time delivery of agent responses. This is not an individual ML engineering role.

You will own an agent orchestration engine, implement lease-based session management, and optimize an event-driven pipeline using SQS, Kafka, gRPC, and Socket.IO, with Python and Go for production-grade concurrency.