Data Engineer (Big Data)

Pacer Group; Toronto; On-site; CAD 90,000 - 130,000; Full time

Job summary

Pacer Group in Toronto, Canada is seeking a data engineer with strong experience in big data processing using Spark, Kafka, and NiFi to design and maintain data pipelines.

You will work with PySpark, Spark SQL/DataFrames, Hadoop components (HDFS, Hive, YARN), and NiFi to ensure reliable real-time data flows and scalable storage.

Qualifications

• Hands-on experience with PySpark and Spark SQL/DataFrames.
• Proficiency with Kafka for real-time data streaming.
• Strong knowledge of the Hadoop ecosystem (HDFS, Hive, YARN) and NiFi.

Skills

SQL proficiency

Python programming

Tools

PySpark

Apache Spark

Spark SQL

DataFrames

Apache Kafka

HDFS

Hive

YARN

Apache NiFi

SQL

Python

Job description

• Strong hands-on experience with PySpark and Apache Spark, including Spark SQL and DataFrames.
• Proficiency in Apache Kafka for real-time data streaming and integration.
• Strong knowledge of the Hadoop ecosystem, including HDFS, Hive, and YARN.
• Hands-on experience with Apache NiFi for data flow and integration.
• Strong SQL and Python programming skills.

EEOC Compliance:

We are an equal opportunity employer, and all qualified applicants will receive consideration for employment.

DISCLAIMER

AI Usage Policy: Pacer Group uses AI to assist in screening applications. Final hiring decisions are made by human recruiters based on qualifications and experience.