Senior Spark & Scala Engineer – Data Pipelines

LTM; Mississauga; On-site; CAD 90,000 - 130,000; Full time

Job summary

LTM in Mississauga, Ontario is seeking a Specialist with 5 to 7 years of experience in Scala and Apache Spark to design, develop, and optimize scalable big data processing solutions within the Hadoop Spark SparkSQL ecosystem. You will design, implement, and optimize data pipelines, collaborating with data scientists, analysts, and architects to deliver robust data processing applications.

Responsibilities include leading development of big data solutions, optimizing Spark jobs for production,

Qualifications

• Design, develop and optimize scalable data processing applications using Scala and Apache Spark.
• Collaborate with cross-functional teams to build robust data pipelines.
• Analyze large datasets to extract actionable business insights.

Responsibilities

• Lead the development and deployment of big data solutions using Scala, Spark and SparkSQL.
• Collaborate with data scientists, analysts and architects to gather requirements.
• Optimize Spark jobs for performance and scalability in production environments.
• Mentor junior team members and provide technical guidance on best practices.
• Perform code reviews to ensure quality and adherence to standards.
• Monitor system performance and address bottlenecks or failures.
• Participate in sprint planning and agile ceremonies.

Skills

Scala

Apache Spark

SparkSQL

Big data

Hadoop ecosystem

Tools

Apache Spark

Hadoop

SparkSQL

Job description

LTM in Mississauga, Ontario is seeking a Specialist with 5 to 7 years of experience in Scala and Apache Spark to design, develop, and optimize scalable big data processing solutions within the Hadoop Spark SparkSQL ecosystem. You will design, implement, and optimize data pipelines, collaborating with data scientists, analysts, and architects to deliver robust data processing applications.

Responsibilities include leading development of big data solutions, optimizing Spark jobs for production,