Software Engineer, Ray Data — Scalable AI Pipelines

Anyscale; San Francisco (CA); On-site; USD 140,000 - 190,000; Full time

Job summary

Anyscale is expanding the Ray Data team in San Francisco, seeking engineers to advance data processing, scale, and performance for AI workloads. You will contribute to a Python-native data processing engine and collaborate with customers to scale Ray in production environments.

Ideal candidates have 3–4 years of relevant experience and a strong background in distributed systems, data processing, and databases.

Qualifications

• At least 3-4 years of relevant work experience.
• Solid background in building scalable and fault‑tolerant distributed systems.
• Experience with data processing and database internals.
• Passionate about large‑scale systems and performance for AI.

Responsibilities

• Improve the performance of Ray Data and multi‑modal batch inference use cases.
• Ensure efficient scaling across different stages of the data pipeline in a heterogeneous environment.
• Build data loading solutions for production training workloads.
• Focus on stability and fault tolerance at high scale.
• Work with customers and AI native companies in scaling their AI workloads.

Skills

Distributed systems

Data processing

Database internals

Performance optimization

Job description

Anyscale is expanding the Ray Data team in San Francisco, seeking engineers to advance data processing, scale, and performance for AI workloads. You will contribute to a Python-native data processing engine and collaborate with customers to scale Ray in production environments.

Ideal candidates have 3–4 years of relevant experience and a strong background in distributed systems, data processing, and databases.