Senior Data Engineer - Big Data & Analytics (AWS/Spark)

Capgemini; Mississauga; On-site; CAD 110,000 - 170,000; Full time

Job summary

Capgemini in Mississauga, Canada seeks a Senior Data Engineer to join agile teams and develop Hadoop-based analytics for the Enterprise Data Lake initiative. You will integrate data across internal and external sources to provide analytical insights and drive critical systems integration.

The role involves leading data engineering efforts in a fast-paced environment, mentoring teammates, and delivering scalable data processing solutions using AWS Big Data technologies and Spark ecosystem.

Qualifications

• Bachelor's degree in a quantitative field and 9+ years of experience.
• Minimum 5 years experience working with big data solutions.
• Hands-on experience with AWS, shell scripting, SQL, Hadoop, Kafka, Spark, and Git.
• Experience with real-time data streaming systems.
• Ability to write abstracted reusable code components.
• Programming in Scala, Java, or Python.

Responsibilities

• Participate in the agile development process.
• Develop functional and technical specifications from business requirements for the commercial platform.
• Ensure application quality and adherence to performance requirements.
• Help create project estimates and plans. Represent engineering team in project meetings and solution discussions.
• Participate code review process.
• Work with team members to achieve business results in a fast paced and quickly changing environment.
• Pair up with data engineers to develop cutting edge Analytic applications leveraging AWS Big Data technologies (Kafka, PySpark, Spark, Hadoop, Shell Scripting, SQL).
• Mentor and influence up and down the chain of command.
• Perform other duties and/or projects as assigned.

Skills

Big data experience

Agile development

Mentoring

Analytical thinking

Education

Bachelor's degree in quantitative field

Tools

AWS

Kafka

Spark

PySpark

Hadoop

Shell scripting

SQL

Git

Job description

Capgemini in Mississauga, Canada seeks a Senior Data Engineer to join agile teams and develop Hadoop-based analytics for the Enterprise Data Lake initiative. You will integrate data across internal and external sources to provide analytical insights and drive critical systems integration.

The role involves leading data engineering efforts in a fast-paced environment, mentoring teammates, and delivering scalable data processing solutions using AWS Big Data technologies and Spark ecosystem.