Data Engineer

LanceSoft, Inc.; Montreal; Vor Ort; CAD 90.000 - 120.000; Vollzeit

Zusammenfassung

Lancesoft in Montreal seeks a data engineering professional to design and optimize data pipelines using Python, PySpark, and Snowflake.

The role requires strong SQL skills, experience with ETL/ELT, and familiarity with cloud platforms and modern data warehousing concepts. Collaboration with analytics teams and CI/CD practices are part of the daily routine.

Qualifikationen

• Strong hands-on experience in Python programming with Pandas, NumPy, and data processing libraries.
• Strong experience with PySpark for distributed data processing and large-scale data transformation.
• Extensive knowledge of SQL, including complex queries, joins, CTEs, window functions, and query optimization.
• Hands-on experience in ETL/ELT development, data integration, data migration, and pipeline orchestration.
• Strong experience with Snowflake, including Snowflake SQL, virtual warehouses, stages, streams, tasks, and Snowpipe.
• Good understanding of relational databases such as SQL Server, Oracle, PostgreSQL, or MySQL.
• Experience in data warehousing concepts, dimensional modeling, fact and dimension tables, and schema design.
• Understanding of batch processing, incremental loads, data validation, and error handling.
• Experience with Git, version control, debugging, and code deployment.

Kenntnisse

Python

PySpark

SQL

ETL/ELT

Snowflake

Relational Databases

Data Warehousing Concepts

Batch Processing

Git

Tools

Apache Spark

Databricks

Apache Airflow

Azure Data Factory

AWS

Azure Data Lake

Snowflake SQL

Jobbeschreibung

• Strong hands-on experience in Python programming, including Pandas, NumPy, and data processing libraries.
• Strong experience with PySpark for distributed data processing and large-scale data transformation.
• Extensive knowledge of SQL, including complex queries, joins, CTEs, window functions, and query optimization.
• Hands-on experience in ETL/ELT development, data integration, data migration, and pipeline orchestration.
• Strong experience with Snowflake, including Snowflake SQL, virtual warehouses, stages, streams, tasks, and Snowpipe.
• Good understanding of relational databases such as SQL Server, Oracle, PostgreSQL, or MySQL.
• Experience in data warehousing concepts, dimensional modeling, fact and dimension tables, and schema design.
• Understanding of batch processing, incremental loads, data validation, and error handling.
• Experience with Git, version control, debugging, and code deployment.

Preferred Skills

• Experience with cloud platforms such such as AWS, Azure, or GCP.
• Knowledge of AWS S3, Azure Data Lake, or similar cloud storage solutions.
• Experience with Apache Spark, Databricks, Apache Airflow, or Azure Data Factory.
• Understanding of CI/CD pipelines and automated deployment processes.
• Exposure to Kafka, real-time data processing, and streaming architectures.
• Knowledge of data governance, metadata management, and data lineage.