Apache Spark Committer - Remote work | REF#308674

BairesDev; Brasil; Remote; BRL 657,000 - 986,000; Full time

Job summary

BairesDev is seeking an experienced Apache Spark-focused software engineer to contribute to core components, optimize query planning, and enhance Catalyst-based execution paths. The role is fully remote, allowing work from anywhere with a strong focus on performance and quality.

You will collaborate with the Spark ecosystem, review code, mentor contributors, and help shape the distributed engine's roadmap while maintaining high documentation and API stability. English proficiency is required.

Qualifications

• 4+ years of experience in software engineering focusing on systems programming or big data internals.
• Proven expertise contributing to Apache Spark codebase or similar high-performance engines.
• Advanced proficiency in Java or C++ and deep knowledge of query engine internals like Spark Catalyst.

Responsibilities

• Contribute high-quality code to Spark core components, focusing on query planners and Catalyst optimizer.
• Perform rigorous code reviews for community PRs to ensure maintainability and API stability.
• Design and implement vectorized execution paths and columnar processing to boost engine throughput.

Skills

Apache Spark

Java/C++

Query planners

Tools

Parquet

Iceberg

Job description

At BairesDev®, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley.

Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works remotely on roles that drive significant impact worldwide.

When you apply for this position, you're taking the first step in a process that goes beyond the ordinary. We aim to align your passions and skills with our vacancies, setting you on a path to exceptional career development and success.

Apache Spark Committer at BairesDev

What You'll Do

• Contribute high-quality code to core Spark components, specifically focusing on query planners and the Catalyst optimizer.
• Perform rigorous code reviews for community pull requests to ensure maintainability, documentation quality, and API stability.
• Design and implement advanced vectorized execution paths and columnar processing features to enhance engine throughput.
• Collaborate with the Apache Software Foundation and global maintainers to define the technical roadmap for Spark's distributed computing engine.
• Monitor project health by resolving critical bugs and identifying performance bottlenecks in large-scale data processing workflows.
• Mentor new contributors and participate in community governance, including voting on releases and major architectural proposals.

What We Are Looking For

• 4+ years of experience in Software Engineering with a focus on systems programming or big data internals.
• Proven expertise in contributing to the Apache Spark codebase or other high-performance query engines.
• Advanced proficiency in Java or C++ and deep understanding of query engine internals like Spark Catalyst.
• Hands-on experience with columnar storage formats such as Parquet or Iceberg and execution engines.
• Strong background in designing query planners, optimizers, and vectorized execution paths.
• Advanced proficiency in English.

How we do make your work (and your life) easier:

• 100% remote work (from anywhere).
• Excellent compensation in USD or your local currency if preferred
• Hardware and software setup for you to work from home.
• Flexible hours: create your own schedule.
• Paid parental leaves, vacations, and national holidays.
• Innovative and multicultural work environment: collaborate and learn from the global Top 1% of talent.
• Supportive environment with mentorship, promotions, skill development, and diverse growth opportunities.

Join a global team where your unique talents can truly thrive and make a significant impact!