Software Engineer, AI Systems & Performance

Cotidal; Palo Alto (CA); On-site; USD 180,000 - 260,000; Full time

Benefits offered by this job

Medical, dental, and vision coverage

Unlimited PTO

Relocation support

Job summary

Cotidal Limited in Palo Alto is seeking an experienced ML systems engineer to define and build Cotidal’s workload software stack for training, reinforcement learning, and inference. You’ll make architectural decisions and drive systems from initial implementation to production-scale as workloads and accelerators evolve.

Working with customers and teammates, you’ll identify problems that matter, set technical priorities, and solve them.

Qualifications

• Built software for training, inference, or accelerated computing and understood behavior under performance pressure and failures.
• Able to navigate from model code to runtime/hardware implications and back to workload impact.

Responsibilities

• Design framework integrations and distributed execution systems for training, RL, and inference.
• Improve performance by optimizing model code, compilers, runtimes, and kernels while maintaining accuracy.
• Own reliability of systems at scale, define validation tests, observability, and recovery mechanisms.

Skills

Training software

Inference software

Distributed execution

Performance optimization

GPU/TPU experience

Software engineering fundamentals

Tools

PyTorch

JAX

XLA

CUDA

Triton

Pallas

Job description

Who We Are

Supercomputing built for AI.

Cotidal is building the computing infrastructure that AI companies and research labs need to train and run their models. Starting with TPUs, we design the hardware and software together around their workloads, with the goal of making powerful AI chips easier to use at scale.

Our mission is endless compute for humanity. Over time, we want customers to choose across chip stacks without rebuilding their infrastructure.

The Role

You’ll define and build Cotidal’s workload software stack for training, reinforcement learning, and inference. You’ll make architectural decisions and carry systems from their first implementation through production scale as models, workloads, and accelerator architectures evolve.

Working directly with customers and teammates, you’ll identify the problems that matter, set technical priorities, and decide how to solve them. We’re looking for engineers with depth in one area, the judgment to navigate open-ended problems, and the curiosity to work across the stack.

What You Will Do

Model & Framework Integration: Design and build the framework integrations and distributed execution systems that bring training, RL, and inference workloads to Cotidal’s accelerators. Extend ML frameworks and libraries, resolve compatibility and compilation issues, and shape support for new model architectures and workflows.

Performance Engineering: Set performance goals, build representative benchmarks, and identify the changes that will have the greatest impact on real workloads. Improve model code, compilers, runtimes, and kernels to increase training efficiency, shorten RL iteration time, and improve inference throughput and latency while preserving correctness and model quality.

Production Reliability: Own the reliability of the systems you build as workloads scale. Define validation and regression tests, observability, checkpointing, and recovery mechanisms. Turn production experience into reusable capabilities and upstream improvements that make new workloads easier to support and existing ones easier to operate.

What We Are Looking For

• You've built software for training, inference, or accelerated computing and stayed close to how it behaves in practice—through performance limits, failures, and changing requirements.

• You follow a problem wherever it leads: from model code and distributed execution into the runtime or hardware, then back to whether the improvement helps the overall workload.

• You have strong software engineering fundamentals and understand how compute, memory, and communication shape system performance. You're comfortable reading unfamiliar code and using experiments to test your understanding.

• You turn open-ended goals into a technical plan, decide what to prioritize, and carry the work through to a reliable system. You take responsibility for the outcome and adjust your approach as you learn.

• You work directly with users and teammates, make your reasoning clear, and seek out the expertise needed to move a problem forward. You have opinions about how systems should be built and have changed your mind when the evidence called for it.

Especially Valuable

Experience in one or more of the following:

• ML frameworks and compilers such as PyTorch, JAX, or XLA, and hands-on work with GPUs, TPUs, Trainium, or other accelerator architectures.

• RL and post-training systems, including rollout generation, weight synchronization, and frameworks such as verl.

• Distributed training and execution, including sharding, model parallelism, collective communication, and overlapping computation with communication.

• Kernel development or compiler optimization using Pallas, Triton, CUDA, or similar tools.

• Inference optimization with vLLM, SGLang, or similar systems, including batching, KV caches, and quantization.

Why Join Cotidal

Close the gap between accelerator capability and useful workload performance through software for training, RL or inference, with responsibility for diagnosis and reliability.

Own work from design through delivery, working directly with the founders. Help shape the foundations of an early company through direct discussion and work across disciplines.

Location & What We Offer

Location: Palo Alto, in person.

Compensation: Competitive salary and equity.

Visa sponsorship: We sponsor work visas and help you navigate the process.

Benefits: Medical, dental, and vision coverage, unlimited PTO, and relocation support as needed.