Principal Data Processing Optimization Architect

NetApp; United States; Presencial; USD 228.000 - 339.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Health Insurance

Life Insurance

Retirement Plans

Paid Time Off

Descripción de la vacante

DataPelago is building an optimized data processing engine to accelerate diverse data workloads, from structured to unstructured, with Spark and Flink integrations. We seek a Senior/Principal Data Processing Optimization Engineer to lead planning and optimization breakthroughs for CPU and GPU deployments.

You will drive research and implementation of advanced query optimization, mentor teammates, and ensure performance in real deployments.

Formación

• 15+ years of experience developing query planning and optimization techniques.
• Experience with parallel platforms such as Apache Spark, Apache Flink, or Presto/Trino.
• Deep knowledge of SQL semantics and SQL processing.
• Bachelor's degree in CS; Master's/PhD preferred.

Responsabilidades

• Research, design, implement, and deliver logical and physical query plan optimization techniques for massively parallel data processing engines.
• Integrate DataPelago acceleration with open-source engines like Spark and Flink.
• Ensure product performance in customer deployments and workloads.
• Mentor junior engineers and drive technical leadership across teams.

Conocimientos

Query planning

SQL processing

Parallel processing

Python processing

Open-source engines

Spark

Flink

Presto/Trino

Educación

Bachelor's degree in computer science

Master's or PhD preferred

Herramientas

Spark

Flink

Presto/Trino

Descripción del empleo

DataPelago is building an optimized data processing engine to accelerate diverse data workloads, from structured to unstructured, with Spark and Flink integrations. We seek a Senior/Principal Data Processing Optimization Engineer to lead planning and optimization breakthroughs for CPU and GPU deployments.

You will drive research and implementation of advanced query optimization, mentor teammates, and ensure performance in real deployments.