Senior Data Engineer - Spark, PySpark & Big Data Architect

Capgemini; Mississauga; Hybrid; CAD 72,000 - 80,000; Full time

Benefits offered by this job

Vacation 12-25 days

Health/dental/vision coverage

RRSP retirement plan

Job summary

Capgemini is seeking an experienced data and analytics professional for a hybrid role based in Mississauga. You will lead end-to-end data processing using HDFS, Spark, PySpark and SQL, collaborating with business users and cross-functional teams.

The role requires 5-8 years of relevant experience, strong knowledge of Spark components, and hands-on programming in Python. Capgemini Canada offers a comprehensive benefits package and opportunities to impact large-scale projects.

Qualifications

• Bachelor's degree in a related field.
• 5-8 years of relevant experience.
• Experience in systems analysis and programming of software applications.
• Experience in managing and implementing successful projects.

Responsibilities

• Monitor and control all phases of development process and analysis design construction testing and implementation as well as provide user and operational support on applications to business users.
• Consult with users, clients and other technology groups on issues, recommend advanced programming solutions and install and assist customer exposure systems.
• Ensure essential procedures are followed and help define operating standards and processes.
• Serve as advisor or coach to new or lower level analysts.
• Has the ability to operate with a limited level of direct supervision.
• Can exercise independence of judgement and autonomy.
• Acts as SME to senior stakeholders and/or other team members.

Skills

HDFS

YARN

MapReduce

Spark Core

PySpark

Spark SQL

Spark Streaming

Spark GraphX

Python

SQL

HBase

NoSQL

Pandas

NumPy

Education

Bachelor's degree

Tools

Zookeeper

Cassandra

MongoDB

Job description

Capgemini is seeking an experienced data and analytics professional for a hybrid role based in Mississauga. You will lead end-to-end data processing using HDFS, Spark, PySpark and SQL, collaborating with business users and cross-functional teams.

The role requires 5-8 years of relevant experience, strong knowledge of Spark components, and hands-on programming in Python. Capgemini Canada offers a comprehensive benefits package and opportunities to impact large-scale projects.