Senior Software Engineer, Agent Infrastructure

Cohere; San Francisco (CA); On-site; USD 180,000 - 240,000; Full time

Benefits offered by this job

Lunch Stipend

Health Insurance

Dental Insurance

Mental Health Benefits

RRSP Matching

401(k)

Pension Scheme

Paid Parental Leave

Annual Enrichment Benefits

Education and Learning Stipend

Paid Vacation

Travel Budget

Co-Working Benefit

Home Office Stipend

Job summary

Cohere in San Francisco is seeking an infrastructure-focused engineer to build production-ready ML infrastructure for agentic AI, including secure execution, identity and authentication, model routing, resource management, and agent state capabilities.

You will prototype and evaluate technologies, develop core platform capabilities, and collaborate with research teams to shape infrastructure for future agent systems.

Qualifications

• Experience building production ML infrastructure.
• Strong systems fundamentals: scalable, fault-tolerant distributed systems.
• Experience with agentic systems and model routing is desirable.

Responsibilities

• Build production-ready infra for agentic AI and secure execution.
• Implement identity/authentication, routing, and resource/state management.
• Prototype technologies and collaborate with research teams to productionize.

Skills

Production ML Infra

Systems Engineering

Agentic AI Systems

Distributed Systems

Kubernetes

Model Routing

LLM Provider Frameworks

Secure Code Execution

Identity and Authentication

Resource Management

State Management

Fault Tolerance

Storage Systems

Filesystem Abstractions

Code Execution Sandboxes

Productionization

Job description

Build production-ready infrastructure for agentic AI, including secure execution, identity and authentication, model routing, resource management, and agent state capabilities. Prototype and evaluate technologies, develop core platform capabilities, and collaborate with research teams to shape infrastructure for future agent systems.

Requirements: Candidates should have experience building production ML infrastructure and strong systems fundamentals, including scalable, fault-tolerant distributed systems and Kubernetes. Relevant experience with agentic systems, model routing, rapid prototyping, and deciding what is ready for production is also sought.

Key Skills: Production ML Infrastructure, Systems Engineering, Agentic AI Systems, Distributed Systems, Kubernetes, Model Routing, LLM Provider Frameworks, Secure Code Execution, Identity and Authentication, Resource Management, State Management, Fault Tolerance, Storage Systems, Filesystem Abstractions, Code Execution Sandboxes, Productionization

Benefits: Lunch Stipend, Health Insurance, Dental Insurance, Mental Health Benefits, RRSP Matching, 401(k), Pension Scheme, Paid Parental Leave, Annual Enrichment Benefits, Education and Learning Stipend, Paid Vacation, Travel Budget, Co-Working Benefit, Home Office Stipend