Python Data Platform Engineer

Apptad; Montreal; On-site; CAD 60,000 - 200,000; Full time

Job summary

Apptad is seeking a Senior Python Data Platform Engineer in Montreal to help build a next-gen data platform for managing data across various systems. The role includes developing a comprehensive data pipeline framework using technologies like Airflow, DBT, and Spark, emphasizing best practices and performance optimization.

Qualifications

• Minimum 10 years in data development with complex environments.
• At least 7 years of experience with SQL and Python.
• Experience with Airflow and Snowflake is a must.

Responsibilities

• Develop components of a unified data pipeline framework in Python.
• Establish best practices for using Airflow, DBT, and Snowflake.
• Assist in testing and deploying the data pipeline framework.

Skills

SQL / PLSQL

Python

Data Pipeline Development

Apache Spark

Snowflake

Airflow

Data Warehousing

Analytical Skills

Problem-Solving

Communication Skills

Education

Bachelor's degree in Computer Science

Tools

Airflow

DBT

Pandas

NumPy

PySpark

Job description

1 week ago Be among the first 25 applicants

Direct message the job poster from Apptad

Delivery Head of Recruitment | IT Talent Acquisition Specialist | Expert in ROI Modeling and Data-Driven Recruitment Strategies

Role Summary :

As a Senior Python Data Platform Engineer, you will be a member of the team with a focus on building our next-gen data platform used for sourcing and storing data from different technology systems across the firm into a centralized data platform that empowers various reporting and analytics solutions. In this role you will be primarily responsible for contributing to the development of a unified data pipeline framework written in Python utilizing technologies such as Airflow, DBT, Spark and Snowflake. You will also be responsible for contributing to the integration of this framework with existing internal platforms for data quality, data cataloging, data discovery, incident logging, and metric generation. You will be working closely with data warehousing leads, data analysts, ETL developers, infrastructure engineers, and data analytics teams to facilitate the implementation of this data platform and data pipeline framework.

Key Responsibilities:

• To develop various components in Python of our unified data pipeline framework.
• To contribute towards the establishment of best practices for the optimal and efficient usage of Airflow, DBT and Snowflake.
• To assist with the testing and deployment of our data pipeline framework utilizing standard testing frameworks and CI/CD tooling.
• To monitor the performance of queries and data loads and perform tuning as necessary.
• To provide assistance and guidance during the QA & UAT phases to quickly confirm the validity of potential issues and to determine the root cause and best resolution of verified issues.

Skills / Qualifications:

• Bachelor's degree in computer science, Software Engineering, Information Technology, or related field required.
• At least 10 years of experience in data development and solutions in highly complex data environments with large data volumes.
• At least 7 years of SQL / PLSQL experience with the ability to write ad-hoc and complex queries to perform data analysis.
• At least 7 years of experience developing data pipelines and data warehousing solutions using Python and libraries such as Pandas, NumPy, PySpark, etc.
• At least 3 years of experience developing solutions in a hybrid data environment (on-Prem and Cloud)
• At least 3 years of experience developing Airflow DAGs to orchestrate data pipelines that utilize branching, dynamic DAG / task generation, and error handling.
• Hands on experience with developing data pipelines for structured, semi-structured, and unstructured data and experience integrating with their supporting stores (e.g. RDBMS, NoSQL DBs, Document DBs, Log Files, etc.)
• Hands on experience with Snowflake is a must.
• Hands on experience with Apache Spark is a must.
• Hands on experience with Airflow (or similar tools like Dagster) a must.
• Hands on experience with DBT preferred.
• Experience with performance tuning SQL queries, Spark job, and stored procedures.
• An understanding of E-R data models (conceptual, logical, and physical).
• Understanding of advanced data warehouse concepts (Factless Fact Tables, Temporal \ Bi-Temporal models, etc.) a plus.
• Strong analytical skills, including a thorough understanding of how to interpret customer business requirements and translate them into technical designs and solutions.
• Strong communication skills both verbal and written. Capable of collaborating effectively across a variety of IT and Business groups, across regions, roles and able to interact effectively with all levels.
• Self-starter. Proven ability to manage multiple, concurrent projects with minimal supervision. Can manage a complex ever changing priority list and resolve conflicts to competing priorities.
• Strong problem-solving skills. Ability to identify where focus is needed and bring clarity to business objectives, requirements, and priorities.

Seniority level

• Seniority level

  Mid-Senior level

Employment type

• Employment type

  Contract

Job function

• Job function

  Business Development

• Industries

  IT Services and IT Consulting, Technology, Information and Media, and IT System Data Services

Referrals increase your chances of interviewing at Apptad by 2x

Sign in to set job alerts for “Python Developer” roles.

Software Developer (Python) - Up to $200k CAD + Huge Bonus

Software Engineer (Python): $220k + Bonus (Elite Fintech)!

Junior Software Engineer (1-5 years) - Python or C++ - Hybrid working - Up to $160,000 CAD Base (+Bonus)

Python Developer (Montreal) – Elite Quant Fund (up to $200K CAD + Bonus + Hybrid)

Montreal, Quebec, Canada CA$60,000.00-CA$200,000.00 5 days ago

Montreal, Quebec, Canada CA$100.00-CA$105.00 23 hours ago

Full Stack Python/React Developer - Hybrid working - $140,000 - $250,000 CAD Base (+Bonus)

Bilingual Python Test Software Developer

Full-Stack Software Developer AI/ML Platform

Montreal, Quebec, Canada $150,000.00-$230,000.00 3 weeks ago

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.