Senior Cloud Data Engineer - Spark & Distributed Systems

Google; Sunnyvale (CA); On-site; USD 174,000 - 252,000; Full time

Job summary

Google Cloud's software engineers develop the next-generation technologies that change how billions of users connect, explore, and interact with information. We're looking for engineers who bring fresh ideas across information retrieval, distributed computing, and large-scale system design to Google Cloud.

This role focuses on Dataproc workloads and Spark-based workloads with opportunities to work on Iceberg/Hudi/Delta Lake, influencing performance, security, and observability.

Qualifications

• Bachelor's degree or equivalent practical experience.
• 5 years of programming experience with Java or C++.
• 3 years of experience testing, maintaining, or launching software products, and 1 year of experience with software design and architecture.
• Experience developing with Spark, Hive, or similar analytical engines.
• Experience designing, analyzing, and troubleshooting large-scale distributed systems.

Responsibilities

• Write and test product or system development code.
• Participate in, or lead design reviews with peers and stakeholders to decide amongst available technologies.
• Review code developed by other developers and provide feedback to ensure best practices.
• Enhance Apache Spark and Lake House technologies like Iceberg, Hudi, or Delta Lake.
• Contribute to and adapt existing documentation or educational content; extend open-source technologies to improve debuggability and observability.

Skills

Programming in Java/C++

Software design and architecture

Distributed systems

Spark/Hive/analytical engines

Education

Bachelor's degree or equivalent practical experience

Master's degree or PhD in Computer Science or related field

Tools

Apache Spark

Apache Hive

Iceberg

Hudi

Delta Lake

Job description

Google Cloud's software engineers develop the next-generation technologies that change how billions of users connect, explore, and interact with information. We're looking for engineers who bring fresh ideas across information retrieval, distributed computing, and large-scale system design to Google Cloud.

This role focuses on Dataproc workloads and Spark-based workloads with opportunities to work on Iceberg/Hudi/Delta Lake, influencing performance, security, and observability.