Senior Data Platform Lead — Delta Lake, Airflow & AWS

Pyramid Consulting, Inc; Región Centro; Híbrido; MXN 900.000 - 1.300.000; Jornada completa

Descripción de la vacante

Pyramid Consulting, Inc is seeking an experienced Data Engineer to drive data platform initiatives in a complex environment. The role emphasizes hands-on work with Databricks, Airflow, Python, SQL and DBT to design reliable data pipelines.

You will implement CI/CD frameworks, govern data catalogs and lineage, and ensure data quality and masking across ecosystems. API integrations and observability with Datadog are also key components.

Formación

• 10+ years of experience in Data Engineering / Data Platform roles.
• Strong hands-on expertise with Databricks, Delta Lake and DAG/workflows.
• Experience implementing CI/CD frameworks (Harness, GitHub Actions, Azure DevOps).
• Deep understanding of data governance, catalogs, lineage and metadata.
• Knowledge of data quality and masking techniques.
• Experience with API-based integrations (e.g., entitlement systems).
• Familiarity with monitoring/observability tools like Datadog.

Responsabilidades

• Lead data engineering initiatives and build scalable data platforms.
• Design and maintain CI/CD pipelines for data workflows.
• Implement data governance, catalogs, lineage and metadata strategies.
• Integrate APIs with entitlement systems and ensure secure data access.
• Monitor and improve data quality and observability across pipelines.

Conocimientos

Databricks

Airflow

Python

SQL

DBT

AWS

CI/CD

Data governance

Data quality

Monitoring

APIs

Herramientas

Databricks

Airflow

Astronomer

Harness

GitHub Actions

Azure DevOps

Datadog

Access Central

Descripción del empleo

Pyramid Consulting, Inc is seeking an experienced Data Engineer to drive data platform initiatives in a complex environment. The role emphasizes hands-on work with Databricks, Airflow, Python, SQL and DBT to design reliable data pipelines.

You will implement CI/CD frameworks, govern data catalogs and lineage, and ensure data quality and masking across ecosystems. API integrations and observability with Datadog are also key components.