Senior Data Processing Optimization Architect

NetApp; San Jose (CA); Hybrid; USD 228,000 - 339,000; Full time

Benefits offered by this job

Health Insurance

401(k) plan

Employee Stock Purchase Plan

Job summary

NetApp is seeking a Senior/Principal Data Processing Optimization Engineer to advance planner and optimization capabilities for DataPelago on CPU and GPU infrastructure. You will influence SQL processing, integration with open-source engines, and performance across varied deployments.

You will collaborate with multiple teams, own major components, and mentor junior engineers while shaping the product's optimization roadmap and customer impact.

Qualifications

• Bachelor’s degree in computer science; Master’s or PhD preferred.
• 15+ years of experience building query planning and optimization for enterprise data processing.
• Deep SQL semantics and processing expertise; Python processing a plus.

Responsibilities

• Design and implement advanced query plan optimization for massively parallel data engines.
• Integrate DataPelago acceleration with Spark, Flink, and other engines.
• Lead metrics for plan quality and performance, mentor junior engineers.

Skills

SQL processing

Query planning

Query optimization

Distributed systems

Spark

Flink

Presto/Trino

Python processing

Collaboration

Communication

Mentoring

Leadership

Education

Bachelor’s degree in Computer Science

Master’s or PhD preferred

Tools

Apache Spark

Apache Flink

Presto/Trino

Job description

NetApp is seeking a Senior/Principal Data Processing Optimization Engineer to advance planner and optimization capabilities for DataPelago on CPU and GPU infrastructure. You will influence SQL processing, integration with open-source engines, and performance across varied deployments.

You will collaborate with multiple teams, own major components, and mentor junior engineers while shaping the product's optimization roadmap and customer impact.