Distributed LLM Inference Engineer - Scale HighThroughput AI

Cerebras; Palo Alto (CA); On-site; USD 120,000 - 160,000; Full time

Benefits offered by this job

Stock Options

Healthcare plans with 99% premium coverage

401k Retirement Plan

Education & Wellbeing Stipend

Paid Parental Leave

Fertility Benefits

Paid Time Off

Commute reimbursement

100% of in-office meals covered

Job summary

Anyscale is seeking a Distributed LLM Inference Engineer in Palo Alto, California. The role focuses on pushing the boundaries of performance for AI inference at large scale, collaborating closely with product teams and open source communities.

The ideal candidate should have experience in running ML inference, familiarity with top deep learning frameworks like PyTorch, and a strong grasp of distributed systems. Attractive benefits and compensation plan included.

Qualifications

• Familiarity with running ML inference at large scale with high throughput and low latency.
• Familiarity with deep learning and deep learning frameworks (e.g. PyTorch).
• Solid understanding of distributed systems and ML inference challenges.

Responsibilities

• Iterate quickly with product teams to deliver end to end solutions for high scale inference.
• Integrate Ray Data and LLM engine for optimizations achieving low cost solutions.
• Contribute to open source software improvements and adoption of techniques.

Skills

Running ML inference at large scale

Deep learning frameworks (e.g., PyTorch)

Distributed systems

Tools

CUDA

vLLM

Job description

Anyscale is seeking a Distributed LLM Inference Engineer in Palo Alto, California. The role focuses on pushing the boundaries of performance for AI inference at large scale, collaborating closely with product teams and open source communities.

The ideal candidate should have experience in running ML inference, familiarity with top deep learning frameworks like PyTorch, and a strong grasp of distributed systems. Attractive benefits and compensation plan included.