Data Engineer

VySystems; Halifax; Presencial; CAD 90 000 - 120 000; Tempo integral

Resumo da oferta

VySystems is seeking a Data Engineer to design, deploy, and manage robust data pipelines across hybrid cloud environments in Halifax. You’ll work with Python, AWS services, CDP and Snowflake to build scalable, secure data solutions for structured and unstructured data.

You will design ETL/ELT workflows on-prem and in the cloud (AWS), optimize data lake architectures on CDP, and ensure data governance and security while enabling advanced analytics with analysts and data scientists.

Qualificações

• Experience designing and deploying ETL/ELT pipelines.
• Strong Python scripting skills and API integration.
• Experience with AWS services (S3, Glue).
• Experience with CDP and Snowflake.
• Data lake design and data modeling forStructured/unstructured data.

Responsabilidades

• Design and implement complex ETL/ELT workflows for on-prem and Cloud platform (AWS).
• Build and optimize data lake architectures on CDP.
• Develop data models in Snowflake and CDP environments.
• Collaborate with analysts to enable advanced analytics on data lake assets.
• Ensure security/governance compliance for sensitive data at scale.

Conhecimentos

Python

AWS

Cross-functional collaboration

Clear communication

Problem-solving

Ferramentas

CDP (Cloudera Data Platform)

Snowflake

S3

GitHub

Azure storage

Descrição da oferta de emprego

Key Requirements

We are seeking a skilled Data Engineer to design, deploy, and manage robust data pipelines across hybrid cloud environments. The ideal candidate will combine expertise in Python, AWS services, and enterprise cloud data platforms to build scalable, secure solutions for structured and unstructured data at enterprise scale.

Key Responsibilities

• Design and implement complex ETL/ELT workflows for diverse data sources both on-prem and Cloud platform (AWS)


• Build and optimize data lake architectures on Cloudera Data Platform (CDP)


• Develop data models in Snowflake cloud warehouse and CDP environments


• Collaborate with analysts to enable advanced analytics on data lake assets


• Ensure security/governance compliance for sensitive data at scale

Technical Expertise Required

• Python: Advanced scripting for data processing, API integration, and automation


• AWS : Amazon S3, AWS Glue


• Data Platforms:


• Cloudera Data Platform (CDP) for Hadoop/Spark ecosystems


• Snowflake cloud data warehouse architecture


• Storage Solutions: Data lake design (S3/ADLS), structured/unstructured data optimization


• Version Control: GitHub collaboration


• Cloud Services: AWS/Azure storage integration with CDP/Snowflake

Soft Skills

• Problem-solv


• ing for hybrid cloud data challenges


• Cross-functional collaboration with data scientists and analysts


• Clear communication of complex technical solutions

Nice-to-Have Qualifications

• Certifications: Cloudera CCP, SnowPro, Red Hat OpenShift


• Experience with streaming platforms (Flink, Kafka Streams)