Machine Learning Engineer, Agentic Systems

ServiceNow; Mountain View (CA); On-site; USD 180,000 - 260,000; Full time

Job summary

ServiceNow in Mountain View, CA is hiring a distributed systems engineer to build the runtime infrastructure powering Moveworks' AI agents, emphasizing orchestration, execution, and real-time delivery of agent responses. This is not an individual ML engineering role.

You will own an agent orchestration engine, implement lease-based session management, and optimize an event-driven pipeline using SQS, Kafka, gRPC, and Socket.IO, with Python and Go for production-grade concurrency.

Qualifications

• Designing distributed systems with strong consistency and idempotency.
• Building asynchronous, event-driven services with robust error handling.
• Experience with message queues (SQS, Kafka) and pub/sub patterns.
• Proficiency with DynamoDB and Redis for state management.
• Skills in observability using OpenTelemetry and metrics collection.
• Experience with gRPC and protobuf-based services.
• 2+ years building production backend or infrastructure systems.
• Proficiency in Python and/or Go, with preference for both.

Responsibilities

• Develop and maintain an agent orchestration engine coordinating long-running sessions across multiple LLM calls.
• Implement lease-based ownership, heartbeats, and crash recovery via checkpoints.
• Operate and optimize SQS FIFO queues, Kafka consumers, and real-time streaming via gRPC/Socket.IO.
• Leverage Python asyncio TaskGroups for concurrent tasks with fail-fast semantics.
• Instrument systems with OpenTelemetry; propagate trace context across async boundaries.
• Manage Redis and DynamoDB state stores with per-org/per-bot scoping.

Skills

Distributed systems

Async programming

Event-driven architectures

Databases/Storage

Observability

gRPC/protobuf

Backend development

Python/Go

Tools

DynamoDB

Redis

SQS

Kafka

Socket.IO

OpenTelemetry

Prometheus

protobuf

gRPC

Job description

This is a distributed systems engineering role at ServiceNow, located onsite in Mountain View, CA. The position focuses on building the runtime infrastructure that powers Moveworks' AI agents, emphasizing orchestration, execution, and real-time delivery of agent responses. It is not an individual ML engineering role.

RESPONSIBILITIES

• Agent orchestration engine: develop and maintain a state machine that manages long running agent sessions, coordinating planning, execution, and user interaction across multiple LLM calls and tool invocations
• Distributed session management: implement lease-based ownership using DynamoDB conditional writes, heartbeat protocols, and crash recovery via checkpointing
• Event driven message pipeline: operate and optimize SQS FIFO queues for ordered delivery, Kafka consumers for event processing, and real time streaming via gRPC and Socket.IO
• Structured concurrency: leverage Python asyncio TaskGroups to run multiple concurrent tasks per session (polling, lease heartbeats, output publishing, orchestrator execution) with fail fast semantics and graceful cancellation
• Observability infrastructure: instrument with OpenTelemetry, propagate distributed trace context across async boundaries, manage custom span lifecycles for sessions that span minutes
• Caching and state layers: manage Redis and DynamoDB KV stores with per-org per-bot scoping, optimize batch reads, and support hot reload configuration

REQUIREMENTS

• Distributed systems design including consistency models, idempotency, exactly once delivery, and distributed locking or leasing
• Concurrent and asynchronous programming: Python asyncio, Go goroutines, structured concurrency, cancellation handling
• Event driven architectures: experience with message queues (SQS, Kafka), pub/sub, backpressure, and delivery guarantees
• Database systems for infrastructure: DynamoDB with conditional writes and transactions, Redis with connection pooling and pub/sub
• Observability: OpenTelemetry, distributed tracing, span context propagation, Prometheus metrics
• gRPC and protobuf: streaming RPCs, service interface design, and robust error handling patterns
• 2+ years building production backend or infrastructure systems
• Strong in Python or Go, with preference for proficiency in both languages
• Experience designing and operating systems capable of handling real traffic at scale
• Ability to navigate ambiguity and solve novel problems without textbook solutions

TECHNOLOGIES

• Python
• Go
• asyncio
• DynamoDB
• Redis
• SQS
• Kafka
• gRPC
• protobuf
• Socket.IO
• OpenTelemetry
• Prometheus

COMPANY DESCRIPTION

Moveworks is an Agentic AI Assistant platform that enables employees to interact with business systems through natural language, driven by advanced large language models and proprietary reasoning capabilities. The solution aims to automate tasks, streamline workflows, and empower AI to take initiative across enterprise processes. Moveworks serves millions of employees at hundreds of organizations, including a significant share of the Fortune 500, and has received recognition across multiple technology and business lists.

WORK PERSONAS

ServiceNow supports flexible work arrangements, with work personas including flexible, remote, or in office. Eligibility is determined by the nature of the work and proximity to a ServiceNow office, potentially verified by distance checks through a third party.

EQUAL OPPORTUNITY EMPLOYER

ServiceNow is an equal opportunity employer. All qualified applicants will receive consideration regardless of race, color, creed, religion, sex, sexual orientation, national origin or ancestry, age, disability, gender identity or expression, marital status, veteran status, or any other protected category.

ACCOMMODATIONS

We strive for an accessible and inclusive candidate experience. If you require a reasonable accommodation to complete any part of the application process, or if you are unable to use the online application and need an alternative method to apply, please contact [e-mail] for assistance.

EXPORT CONTROL REGULATIONS

Positions that require access to controlled technology subject to export control regulations may require approval from government authorities. Employment is contingent upon ServiceNow obtaining any necessary export licenses or approvals.