Senior Data Processing Engineer (OSS) – Open-Source Core

NetApp; San Jose (CA); Hybrid; USD 196,000 - 293,000; Full time

Benefits offered by this job

Health Insurance

Life Insurance

Retirement or Pension Plans

Paid Time Off

various Leave options

Performance-Based Incentives

employee stock purchase plan

restricted stocks (RSU’s)

Job summary

NetApp is seeking a Senior/Principal Data Processing Engineer - OSS to advance open-source platforms like Apache Gluten, Velox, Spark and Flink within DataPelago’s engine. You will design, implement and optimize high-performance data processing components for large-scale deployments.

You will shape architecture, collaborate with open-source communities, and drive improvements in performance, reliability and scalability across Linux environments.

Qualifications

• Bachelor's degree in Computer Science or related field (or Master's).
• 12+ years of relevant experience in data processing engines.
• 3+ years in developing core components of Spark, Flink, Doris, Gluten, Velox, or equivalents.
• 3+ years optimizing performance of data processing engines.
• Strong knowledge of Spark, Flink, Presto/Trino architectures.
• Proven release of high-performance engines for large deployments.

Responsibilities

• Architect interface with OSS platforms and engines.
• Design improvements to Gluten and Velox integration.
• Develop, test, optimize, and maintain engine components.
• Analyze tech roadmap for differentiation.
• Collaborate with engineering, product, OSS communities, and customers.
• Drive CI/CD and code quality practices.

Skills

C/C++/Java programming

Linux development

Performance optimization

Spark/Flink experience

Open-source collaboration

Education

Bachelor's or Master's in CS

Tools

Apache Gluten

Velox

Apache Spark

Apache Flink

Presto/Trino

Linux

Job description

NetApp is seeking a Senior/Principal Data Processing Engineer - OSS to advance open-source platforms like Apache Gluten, Velox, Spark and Flink within DataPelago’s engine. You will design, implement and optimize high-performance data processing components for large-scale deployments.

You will shape architecture, collaborate with open-source communities, and drive improvements in performance, reliability and scalability across Linux environments.