Remote Data Engineer: Big Data Pipelines

Capgemini Engineering; Canada; À distance; CAD 72 000 - 139 000; Plein temps

Avantages offerts par ce poste

Paid time off

Medical, dental, and vision coverage

Retirement savings plans

Life and disability insurance

Employee assistance programs

Résumé du poste

Capgemini Engineering in Canada is seeking a highly skilled Software/Data Engineer to build scalable data processing solutions and maintain data pipelines using Apache, SQL, Python, and Scala.

You will optimize ETL workloads, work with Parquet/Hive/HDFS, and collaborate across data, software, and product teams to deliver data-driven solutions. Remote work in Canada with competitive compensation and benefits.

Qualifications

• Bachelor’s or master’s degree in computer science, Engineering, or a related technical field.
• 5+ years of professional software engineering or data engineering experience.
• Strong hands-on experience with: Apache /SQL/Python/Scala.
• Strong understanding of Hadoop ecosystem technologies, including: Hive/Parquet/HDFS
• Experience developing and optimizing ETL pipelines processing large-scale datasets.
• Strong SQL programming and query optimization skills, familiarity with Linux environments and shell scripting.
• Experience using version control systems such as Git.
• Preferred qualifications include experience with AWS cloud platforms, distributed computing and large-scale data architecture, Airflow orchestration, data lake and lakehouse environments, data storage formats such as Iceberg, ORC, and Avro, and containerization technologies including Docker and Kubernetes.

Responsabilités

• Design, develop, and maintain scalable data processing applications using Apache and SQL.
• Build high-performance ETL and data transformation pipelines using Python and Scala.
• Develop and optimize batch and near-real-time data processing workflows.
• Work with large-scale data sets stored in Parquet, Hive, and distributed storage systems.
• Write efficient SQL queries and optimize execution plans for performance and scalability.
• Design data models, partitioning strategies, and storage layouts to improve processing efficiency.
• Collaborate with Data Engineers, Software Engineers, Data Scientists, and Product Teams to deliver data-driven solutions.
• Implement automated testing, monitoring, and deployment processes for data applications.
• Ensure data quality, reliability, security, and governance standards are met.

Connaissances

Apache/SQL/Python/Scala

Hadoop ecosystem (Hive/Parquet/HDFS)

SQL optimization

Linux & shell scripting

Git version control

Team collaboration

Cloud (AWS)

Airflow orchestration

Docker & Kubernetes

Formation

Bachelor's or Master’s in CS/Engineering

Outils

Docker

Kubernetes

Airflow

Iceberg

ORC

Avro

Description du poste

Capgemini Engineering in Canada is seeking a highly skilled Software/Data Engineer to build scalable data processing solutions and maintain data pipelines using Apache, SQL, Python, and Scala.

You will optimize ETL workloads, work with Parquet/Hive/HDFS, and collaborate across data, software, and product teams to deliver data-driven solutions. Remote work in Canada with competitive compensation and benefits.