Senior Spark Engineer: Open-Source & Distributed Systems

Remote Jobs

United States

Remote

USD 140,000 - 180,000

Full time

2 days ago

Be an early applicant

Application generator

A complete application in a minute — tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Remote Jobs in the United States is seeking a Senior Software Engineer with deep expertise in Apache Spark to work on Spark internals, contribute to the open-source community, and build scalable data processing solutions aligned with platform requirements. You will tackle complex performance challenges and help shape the platform's data processing capabilities.

The role emphasizes upgrading Spark releases, optimizing internals for performance and reliability, and collaborating with product and

Qualifications

Strong experience with Apache Spark internals and distributed systems.
Hands-on experience contributing to open-source projects.
Strong Java and/or Scala programming skills.
Expertise in query execution, distributed processing, and performance optimization.
Strong debugging and problem-solving abilities.
Ability to work independently and drive technical initiatives.

Responsibilities

Contribute fixes and enhancements to the Apache Spark open-source project.
Upgrade and maintain compatibility with newer Spark releases.
Optimize Spark internals for performance, scalability, and reliability.
Design and develop distributed data processing solutions.
Debug complex Spark-related performance and stability issues.
Collaborate with product and platform teams on architecture and technical initiatives.