Developer (Spark and Scala Exp)

Confiar Services; Toronto; On-site; CAD 110,000 - 150,000; Full time

Job summary

Confiar Services in Toronto, Canada seeks a Spark with Scala Developer with a strong Java background to design, develop, and optimize large-scale data processing applications. The candidate will transition from Java to Scala-based Spark development, building high-performance, scalable data solutions.

The role emphasizes designing and implementing ETL pipelines, optimizing distributed computing, and collaborating with data engineers, architects, and stakeholders to ensure robust and reliable

Qualifications

• 5+ years of software development experience.
• Hands-on Java development
• Experience with Apache Spark and Scala required.

Responsibilities

• Design and build large-scale data processing applications using Spark and Scala.
• Develop and optimize ETL/data pipelines for big datasets.
• Create scalable, high-performance data solutions on distributed platforms.
• Leverage Java expertise to enhance Spark applications and integrate libraries.
• Troubleshoot performance issues and optimize Spark jobs.

Skills

Java development

Apache Spark

Scala

Distributed computing

JVM tuning

SQL

Cloud platforms

Big data

Tools

Databricks

Kafka

Git

CI/CD

Job description

Job Title: Spark with Scala Developer (Java Background)

Location: Toronto, Canada

Duration: Long Term Contract

Job Summary

We are looking for an experienced Spark with Scala Developer with a strong Java background to design, develop, and optimize large-scale data processing applications. The ideal candidate will leverage their Java expertise and transition seamlessly into Scala-based Spark development, building high-performance, scalable, and reliable data solutions.

Key Responsibilities

• Design and develop big data applications using Apache Spark and Scala.
• Build and optimize ETL/data processing pipelines for large datasets.
• Develop scalable, high-performance solutions on distributed data platforms.
• Utilize Java expertise to enhance Spark applications and integrate existing Java libraries.
• Troubleshoot performance issues and optimize Spark jobs.
• Work closely with data engineers, architects, and business stakeholders.
• Ensure code quality through testing, code reviews, and best practices.
• Participate in system design, performance tuning, and production support.

Required Skills

• 5+ years of software development experience.
• Strong hands-on experience with Java development.
• Experience with Apache Spark and Scala.
• Good understanding of distributed computing concepts.
• Knowledge of JVM internals, memory management, and performance tuning.
• Experience with SQL and large-scale data processing.
• Familiarity with Hadoop ecosystem and cloud platforms is a plus.
• Strong problem-solving and analytical skills.

Preferred Qualifications

• Experience with Databricks, Kafka, or streaming technologies.
• Exposure to AWS, Azure, or Google Cloud platforms.
• Knowledge of CI/CD, Git, and Agile methodologies.