Spark Developer (With Java)

Capgemini; Mississauga; On-site; CAD 90,000 - 130,000; Full time

Job summary

Capgemini in Mississauga seeks a Spark Developer with strong Java expertise to design, develop, and optimize large-scale data processing applications. The role emphasizes Apache Spark, Core Java, distributed computing, and related big data technologies to build scalable data solutions.

You will collaborate with data engineers, architects, and business stakeholders to develop batch and near real-time pipelines, optimize performance, and implement best practices across coding, testing, deployment,

Qualifications

• 4+ years of software development experience.
• 2+ years hands-on experience with Apache Spark and Java.
• Bachelor's or Master's degree in Computer Science, Information Technology, or related field.

Responsibilities

• Design, develop, and maintain scalable data processing applications using Apache Spark and Java.
• Develop batch and near real-time data pipelines for large datasets.
• Optimize Spark jobs for performance, memory management, and resource utilization.
• Write efficient and reusable code using Java 8+ features.
• Work with distributed data processing frameworks and big data ecosystems.
• Collaborate with cross-functional teams to gather requirements and deliver data solutions.
• Troubleshoot production issues and provide performance tuning recommendations.
• Implement best practices for coding, testing, deployment, and monitoring.
• Participate in code reviews and technical design discussions.
• Ensure data quality, reliability, and scalability of applications.

Skills

Core Java

Apache Spark

Distributed computing

Hadoop Ecosystem

SQL

Git

CI/CD

Linux/Unix

REST APIs

Microservices

Education

Bachelor's or Master's in Computer Science / IT

Tools

Hadoop

CI/CD tools

Linux

Job description

We are seeking a highly skilled Spark Developer with strong Java expertise to design, develop, and optimize large-scale data processing applications. The ideal candidate will have hands‑on experience with Apache Spark, Core Java, distributed computing, and big data technologies. The candidate will work closely with data engineers, architects, and business stakeholders to build scalable and high‑performance data solutions.

Key Responsibilities

• Design, develop, and maintain scalable data processing applications using Apache Spark and Java.
• Develop batch and near real‑time data pipelines for large datasets.
• Optimize Spark jobs for performance, memory management, and resource utilization.
• Write efficient and reusable code using Java 8/11 features.
• Work with distributed data processing frameworks and big data ecosystems.
• Collaborate with cross‑functional teams to gather requirements and deliver data solutions.
• Troubleshoot production issues and provide performance tuning recommendations.
• Implement best practices for coding, testing, deployment, and monitoring.
• Participate in code reviews and technical design discussions.
• Ensure data quality, reliability, and scalability of applications.

Required Skills

• Strong programming experience in Core Java (Java 8 or above).
• Hands‑on experience with Apache Spark (Spark Core, Spark SQL, DataFrames, Dataset APIs).
• Good understanding of distributed computing concepts.
• Experience with HDFS, Hive, Hadoop Ecosystem.
• Strong SQL skills and experience with relational databases.
• Experience with data ingestion and transformation frameworks.
• Familiarity with Linux/Unix environments.
• Knowledge of REST APIs and microservices architecture.
• Experience with Git and CI/CD processes.

Preferred Skills

• Experience with Spark Streaming, Kafka, Scala, or PySpark.
• Exposure to cloud platforms such as AWS, Azure, or GCP.
• Experience with Airflow, Oozie, or similar workflow orchestration tools.
• Understanding of Delta Lake, Iceberg, or Data Lake architectures.
• Knowledge of containerization tools such as Docker and Kubernetes.

Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, or related field.
• 4+ years of software development experience.
• 2+ years of hands‑on experience developing applications using Apache Spark and Java.