Spark Big Data Engineer | Scala/Java & ETL Analytics

Tata Consultancy Services Limited; Toronto; On-site; CAD 90,000 - 110,000; Full time

Job summary

Tata Consultancy Services Canada Inc. is seeking a Big Data Engineer to design and maintain robust data pipelines using Apache Spark, Scala, and Java for enterprise analytics.

You will ingest, transform, and optimize large datasets, focusing on performance and cost efficiency while collaborating with data scientists and infrastructure teams. You will also implement testing strategies, maintain documentation, and contribute to code quality through CI/CD practices.

Qualifications

• Demonstrated professional experience developing production-grade big data applications.
• Strong expertise in Apache Spark (Core, SQL, Streaming).
• Advanced programming skills in Scala and/or Java.
• Strong understanding of distributed computing principles and architecture.
• Experience with data warehousing and SQL.
• Knowledge of Git and CI/CD pipelines.
• Strong analytical and problem-solving skills.
• Excellent communication and teamwork skills.

Responsibilities

• Design and implement end-to-end data pipelines using Apache Spark for ETL and analytics workloads.
• Develop clean, efficient, and maintainable code using Scala and Java.
• Build solutions to ingest, transform, and aggregate large-scale financial and operational data.
• Analyze and optimize Spark applications for performance, memory efficiency, and cost-effectiveness.
• Collaborate with data scientists, analysts, and infrastructure teams to translate requirements into technical solutions.
• Troubleshoot and resolve production issues within big data systems.
• Implement testing strategies and maintain code quality standards.
• Create and maintain technical documentation and knowledge base articles.

Skills

Scala

Java

SQL

ETL/ELT

Distributed Systems

Performance Tuning & Optimization

Testing Frameworks & Methodologies

Tools

Kafka

Flink

Hadoop

Hive

HBase

Docker

Kubernetes

Job description

Tata Consultancy Services Canada Inc. is seeking a Big Data Engineer to design and maintain robust data pipelines using Apache Spark, Scala, and Java for enterprise analytics.

You will ingest, transform, and optimize large datasets, focusing on performance and cost efficiency while collaborating with data scientists and infrastructure teams. You will also implement testing strategies, maintain documentation, and contribute to code quality through CI/CD practices.