Principal Data Processing Engineer — OSS & Hybrid
NetApp, Inc.
An application made for this job — a tailored resume and cover letter that speak straight to the posting.
Benefits offered by this job
Job summary
NetApp seeks a Senior/Principal Data Processing Engineer - OSS to advance open-source platforms like Apache Gluten, Velox, Spark, and Flink within the DataPelago engine. You will enhance architecture, performance, and reliability, contributing upstream and downstream to scale production deployments.
You will engage with OSS communities, design and implement high-performance components, and drive improvements across CI/CD, testing, and code reviews.
Qualifications
- 15+ years of relevant experience in data processing and OSS.
- 3+ years in developing components for Spark, Flink, or related platforms.
- Experience optimizing the performance of data processing engines.
- Strong knowledge of Spark, Flink, Presto/Trino.
- Excellent C/C++ and Java programming skills.
Responsibilities
- Architect integration with open-source platforms.
- Lead design of performance enhancements with Glue/Velox and integration with our engine.
- Core development of the data processing engine components.
- Analyze technology roadmaps to identify differentiation opportunities.
- Collaborate with engineering, product, OSS communities and customer success teams.
- Drive CI/CD, testing, and code reviews to ensure high quality.
Skills
Education
Tools
Job description
NetApp seeks a Senior/Principal Data Processing Engineer - OSS to advance open-source platforms like Apache Gluten, Velox, Spark, and Flink within the DataPelago engine. You will enhance architecture, performance, and reliability, contributing upstream and downstream to scale production deployments.
You will engage with OSS communities, design and implement high-performance components, and drive improvements across CI/CD, testing, and code reviews.
'/%3e%3cpath%20d='M101.744%2012.4726L102.894%2016.1947L99.0021%2013.2201L101.744%2012.4726ZM105.3%208.40937H100.488L99.0026%203.59473L97.5116%208.41047L92.7%208.40385L96.5965%2011.3834L95.1055%2016.1942L99.0021%2013.2196L101.408%2011.3834L105.3%208.40992V8.40937Z'%20fill='white'/%3e%3cdefs%3e%3clinearGradient%20id='paint0_linear_5593_1381'%20x1='90'%20y1='9.44471'%20x2='99'%20y2='9.44471'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%2300AD70'/%3e%3cstop%20offset='0.998594'%20stop-color='%2300B67A'/%3e%3cstop%20offset='1'%20stop-color='%23DCDCE6'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e){kind=small}
