Software Engineer, Ray Data Platform

Carbon Data Solutions; San Francisco (CA); Presencial; USD 150.000 - 195.000; Jornada completa

Descripción de la vacante

Anyscale is seeking a software engineer for the Ray Data team to scale data processing pipelines and inference workloads across diverse environments, from laptops to clusters.

You will optimize performance, ensure fault tolerance, and collaborate with AI-native companies to deploy scalable ML pipelines. The role requires 3–4 years of relevant experience and a strong background in distributed systems and data processing.

Formación

• 3-4 years of relevant work experience.
• Solid background in building scalable and fault-tolerant distributed systems.
• Experience with data processing, database internals.
• Passionate about large scale systems and performance for AI.

Responsabilidades

• Improve the performance of Ray Data and multi-modal batch inference use cases.
• Ensure efficient scaling across different stages of the Data pipeline in a heterogeneous environment.
• Build data loading solutions for production training workloads.
• Focus on stability and fault tolerance at high scale.
• Work with customers and AI-native companies in scaling their AI workloads.

Conocimientos

Distributed systems

Data processing

Performance optimization

Python

Ray Data

Herramientas

Ray Data

Python

Descripción del empleo

Anyscale is seeking a software engineer for the Ray Data team to scale data processing pipelines and inference workloads across diverse environments, from laptops to clusters.

You will optimize performance, ensure fault tolerance, and collaborate with AI-native companies to deploy scalable ML pipelines. The role requires 3–4 years of relevant experience and a strong background in distributed systems and data processing.