Principal Data Engineer

Confiz; Región Centro; On-site; MXN 900,000 - 1,500,000; Full time

Job summary

Confiz is seeking a Principal Data Engineer with 5+ years of hands-on experience in the modern cloud data stack. The role focuses on building scalable data solutions using Databricks, PySpark, DBT, and Snowflake.

You will lead a team of data engineers, drive architectural decisions, and ensure data quality and pipeline observability in an agile environment. Strong Python/SQL skills and cloud platform experience are required.

Qualifications

• Bachelor’s degree in Computer Science or related field.
• Experience with Databricks and PySpark for large-scale data processing.
• Proficiency with DBT development and CI/CD integration.
• Hands-on with Astronomer/Airflow for orchestration and Cosmos integration.
• Strong Snowflake data modelling, performance tuning, and cost governance.
• Proficient in Python and SQL; modular, tested code and code reviews.

Responsibilities

• Build scalable data pipelines using DBT and Astronomer/Airflow.
• Develop modular DBT transformations (staging → mart) with testing and docs.
• Orchestrate end-to-end workflows with Cosmos-triggered DBT models.
• Design and optimize Snowflake schemas and queries for analytics.
• Enforce data quality and observability in pipelines.
• Write clean Python and SQL; participate in code reviews.
• Lead and mentor a team of data engineers; contribute to sprint planning.

Skills

Leadership

Mentoring

Agile & CI/CD mindset

Data engineering

Education

Bachelor's in Computer Science

Tools

Databricks

PySpark

DBT

Astronomer / Airflow

Cosmos

Snowflake

Python

SQL

Azure / AWS

Job description

Confiz is seeking a Principal Data Engineer with 5+ years of hands-on experience in the modern cloud data stack. This role is ideal for someone passionate about building robust, scalable data solutions and who thrives in agile, collaborative environments.

The core technical stack centers around Databricks / PySpark for large-scale data processing, DBT for transformation, Astronomer / Airflow for orchestration (with Cosmos to run DBT DAGs natively in Airflow), and Snowflake as the cloud data warehouse.

Responsibilities

• Build and maintain scalable data pipelines using DBT and Astronomer/Airflow for batch and incremental data processing.
• Develop modular DBT transformation layers (staging → intermediate → mart) with robust testing, documentation, and incremental model strategies.
• Orchestrate end-to-end workflows on Astronomer / Airflow, using Cosmos to trigger and manage DBT models as native Airflow DAGs.
• Design and optimize Snowflake schemas, queries, and cost controls for analytical workloads.
• Enforce data quality standards through DBT tests and pipeline observability tooling.
• Write clean, well-tested Python and SQL code and actively participate in code reviews.
• Lead and mentor a team of data engineers; contribute to sprint planning, architectural decisions, and technical standards.

Qualifications

• Bachelor’s in Computer Science or a related discipline.
• Databricks & PySpark: Delta Lake, batch and streaming workloads, cluster and cost optimization.
• DBT (Core & Cloud): Advanced data modelling patterns, incremental strategies, custom tests, and CI/CD integration.
• Astronomer / Airflow: DAG authoring, Cosmos for DBT orchestration, and pipeline observability.
• Snowflake: Data modelling, query performance tuning, access control, and cost governance.
• Python and SQL: Strong programming skills with a focus on modular, maintainable, and testable code.
• Cloud platform: Hands-on experience with Azure or AWS.
• Agile & CI/CD: Comfortable working in agile teams with a CI/CD development mindset.

We have a global team of amazing individuals working on highly innovative enterprise projects & products. Our customer base includes Fortune 100 retail and CPG companies, leading store chains, fast growth fintech, and multiple Silicon Valley startups.

What makes Confiz stand out is our focus on processes and culture. Confiz isISO 9001:2015(QMS),ISO 27001:2022(ISMS),ISO 20000-1:2018(ITSM) andISO 14001:2015(EMS) Certified. We have a vibrant culture of learning via collaboration and makingworkplace fun.

People who work with us work with cutting-edge technologies while contributing success to the company as well as to themselves.

To know more about Confiz Limited, visit https://www.linkedin.com/company/confiz/