Intermediate Data Engineer

NES Fircroft; Calgary; On-site; CAD 80,000 - 110,000; Full time

Job summary

A leading company in data solutions is seeking an Intermediate Data Engineer to design scalable data architectures and optimize workflows. The ideal candidate will have hands-on experience in cloud environments and data pipeline development, contributing to innovative data solutions within a collaborative team.

Qualifications

• 1–3 years of hands-on experience with PySpark.
• 3+ years of Python scripting and automation.
• 2+ years working within AWS and/or Azure ecosystems.

Responsibilities

• Develop and maintain scalable data pipelines using PySpark.
• Design and manage serverless ETL/ELT workflows with AWS Glue.
• Write and optimize complex SQL queries for data analysis.

Skills

PySpark

AWS Glue

Python

SQL

Data Pipeline Development

Cloud Platforms

Data Modeling

ETL/ELT Workflow Design

Version Control (Git)

Big Data Tools

Analytical Thinking

Job description

Intermediate Data Engineer

We are actively seeking a talented Intermediate Data Engineer who is passionate about designing scalable data architectures and optimizing complex data workflows. This role is ideal for someone with hands-on experience in cloud environments, robust data pipeline development, and modern data engineering frameworks. You will be joining a forward-thinking team committed to innovation, collaboration, and delivering high-quality data solutions.

Key Responsibilities:

• Develop, optimize, and maintain scalable data pipelines using PySpark and big data ecosystems.
• Design, implement, and manage serverless ETL/ELT workflows using AWS Glue and related AWS services.
• Collaborate with teams to integrate Microsoft Fabric (unified analytics and lakehouse architecture) where applicable.
• Create reusable data transformation scripts and applications using Python.
• Write and maintain complex SQL queries for data analysis and transformation across various relational databases.
• Partner with stakeholders to understand requirements and design efficient data models and end-to-end data integration solutions.
• Ensure best practices in version control, data quality, and workflow automation.

Must-Have Skills & Experience:

• PySpark: 1–3 years of hands-on experience.
• AWS Glue: 1–3 years with serverless data processing.
• Python: 3+ years with strong scripting and automation capabilities.
• SQL: 3+ years of writing and optimizing complex queries.
• Data Pipeline Development: 2–4 years of experience.
• Cloud Platforms: 2+ years working within AWS and/or Azure ecosystems.
• Data Modeling: 1–2 years designing scalable and efficient schemas.
• ETL/ELT Workflow Design: 2–3 years with orchestration and automation.
• Version Control (Git): 1–2 years using Git in collaborative projects.
• Big Data Tools (e.g., Hadoop, Spark): 1–2 years with distributed data systems.
• Analytical Thinking: Strong skills in debugging, performance tuning, and workflow optimization.

Bonus Points:

• Exposure to or foundational knowledge of Microsoft Fabric is a strong asset.