Apache Spark Developer

BairesDev; Recife; On-site; BRL 601,000 - 902,000; Full time

Benefits offered by this job

Competitive compensation in USD or USD

Parental leave & holidays

Multicultural work environment

Global collaboration with top talent

Mentorship & growth opportunities

Job summary

BairesDev is seeking an Apache Spark Developer to work on distributed data processing systems and real-time analytics. You will contribute to Apache Spark and related ecosystem projects (Kafka, Hive, Iceberg), focusing on performance optimization and scalable architecture.

Join a team of the global Top 1% of tech talent, collaborating with engineering teams to deploy robust data platforms. English proficiency required for international collaboration.

Qualifications

• 10+ years of experience in software development.
• Strong expertise in distributed computing and large-scale data processing systems.
• Understanding of Apache Spark architecture, including RDDs, DataFrames, and streaming.
• Proficiency in Scala, Java, and Python (primary Spark development languages).
• Experience in performance tuning of distributed systems at scale.
• Core contributions to Apache open source projects (Spark, Kafka, Hive, or similar).
• Advanced level of English.

Responsibilities

• Contribute actively to Apache Spark and related open source projects as a core contributor.
• Develop and optimize distributed data processing engines for batch and real-time analytical workloads.
• Develop streaming analytics capabilities using Spark Structured Streaming and integration with Apache Kafka.
• Enhance Spark SQL catalyst optimizer and distributed query execution systems.
• Collaborate with engineering teams on system integration and deployment.

Skills

Distributed computing

Spark architecture

Scala

Java

Python

Performance tuning

Open source contributions

English

Tools

Apache Spark

Kafka

Hive

Iceberg

Job description

At BairesDev®, we've been leading the way in technology projects for over 15 years. We deliver cutting-edge solutions to giants like Google and the most innovative startups in Silicon Valley. Our diverse 4,000+ team, composed of the world's Top 1% of tech talent, works on roles that drive significant impact worldwide.

When you apply for this position, you're taking the first step in a process that goes beyond the ordinary. We aim to align your passions and skills with our vacancies, setting you on a path to exceptional career development and success.

Apache Spark Developer at BairesDev

We are seeking an Apache Spark Developer to work on distributed data processing systems and real-time analytics. You'll contribute to Apache Spark and related ecosystem projects (Kafka, Hive, Iceberg), focusing on performance optimization and scalable architecture.

What You’ll Do:

• - Contribute actively to Apache Spark and related open source projects as a core contributor.
• - Develop and optimize distributed data processing engines for batch and real-time analytical workloads.
• - Develop streaming analytics capabilities using Spark Structured Streaming and integration with Apache Kafka.
• - Enhance Spark SQL catalyst optimizer and distributed query execution systems.
• - Collaborate with engineering teams on system integration and deployment.

What we are looking for:

• - 10+ years of experience in software development.
• - Strong expertise in distributed computing and large-scale data processing systems.
• - Understanding of Apache Spark architecture, including RDDs, DataFrames, and streaming.
• - Proficiency in Scala, Java, and Python (primary Spark development languages).
• - Experience in performance tuning of distributed systems at scale.
• - Core contributions to Apache open source projects (Spark, Kafka, Hive, or similar).
• - Advanced level of English.

How we make your work (and your life) easier:

• - Excellent compensation in USD or your local currency if preferred.
• - Paid parental leave, vacation, & national holidays.
• - Innovative and multicultural work environment.
• - Collaborate and learn from the global Top 1% of talent in each area.
• - Supportive environment with mentorship, promotions, skill development, and diverse growth opportunities.

Join a global team where your unique talents can truly thrive and make a significant impact!