Data Platform Engineer: Build Pipelines (Python,GCP)

Confiz; Monterrey; Presencial; MXN 900.000 - 1.300.000; Jornada completa

Descripción de la vacante

Confiz LATAM is seeking a Data Platform Engineer based in Mexico to design, develop, and maintain scalable data orchestration platforms. You will implement ETL/ELT pipelines, build self‑service capabilities for engineers and data scientists, and integrate diverse data sources using Kafka, BigQuery, GCS, and more.

The role emphasizes IaC with Terraform, Airflow orchestration, and Kubernetes, enabling batch and real‑time processing. Strong Python, SQL, and cloud data expertise are essential.

Formación

• Bachelor’s or Master’s degree in Computer Science, Engineering, or equivalent practical experience.
• 5–7 years of professional experience in Data Engineering, Data Platform Engineering, or related software engineering roles.
• Strong professional programming experience with Python, including building and maintaining automation, services, and/or command‑line tools.
• Strong proficiency in SQL, particularly within modern cloud data environments such as BigQuery.
• Hands‑on experience designing and maintaining ETL/ELT pipelines and a solid understanding of modern data pipeline architecture.
• Experience with modern data transformation frameworks such as dbt.
• Strong hands‑on experience with Google Cloud Platform (GCP) data technologies, particularly BigQuery, GCS, and Dataproc Serverless.
• Strong experience with Infrastructure as Code (IaC), particularly Terraform, for provisioning and managing cloud data infrastructure.
• Hands‑on experience building and orchestrating data pipelines using Apache Airflow.
• Experience working with Kubernetes in data platform or cloud environments.
• Experience with streaming and event‑driven data technologies such as Kafka / Confluent Cloud.
• Working knowledge of data governance, metadata management, data cataloging, and data lineage concepts.
• Familiarity with CI/CD and Git‑based development workflows, including tools such as GitHub Actions.
• Experience using AI‑assisted developer tooling, including LLMs, Model Context Protocol (MCP), and/or copilot‑style productivity tools.
• Excellent problem‑solving and communication skills, with the ability to collaborate effectively across engineering and business teams.

Responsabilidades

• Design, evolve, and maintain scalable data orchestration platforms that simplify ETL/ELT operations and abstract the complexities of big data technologies.
• Build and maintain self-service platform capabilities for engineers, data scientists, and analysts.
• Develop and maintain integrations with diverse data sources and destinations, including Kafka, GCS, BigQuery, Postgres, and S3, supporting both batch and real‑time processing.
• Provision and manage cloud data infrastructure using Terraform, leveraging automated and version‑controlled IaC pipelines.
• Build, maintain, and orchestrate reliable data pipelines using Airflow and Kubernetes.
• Develop platform tooling, automation, services, and CLIs using Python.
• Build and operate metadata management, data cataloging, governance, and data‑lineage capabilities to make enterprise data discoverable and trustworthy.
• Design scalable and resilient solutions capable of handling large data volumes, high velocity, and diverse data formats.
• Apply modern CI/CD and Git‑based workflows to platform and data pipeline delivery.
• Evaluate and incorporate emerging big data, ETL, and AI‑assisted development technologies where appropriate.
• Create and maintain comprehensive technical and user documentation for platform capabilities and workflows.

Conocimientos

Python programming

SQL proficiency

ETL/ELT pipelines

dbt

GCP

Airflow

Terraform

Kubernetes

Kafka

Data governance

CI/CD

AI-assisted tooling

Communication

Educación

Bachelor’s or Master’s degree in CS/Engineering

Herramientas

Terraform

Airflow

BigQuery

GCS

Dataproc Serverless

Kubernetes

Kafka

Confluent Cloud

Looker/LookML

DataHub/Collibra/Amundsen

GitHub Actions

Descripción del empleo

Confiz LATAM is seeking a Data Platform Engineer based in Mexico to design, develop, and maintain scalable data orchestration platforms. You will implement ETL/ELT pipelines, build self‑service capabilities for engineers and data scientists, and integrate diverse data sources using Kafka, BigQuery, GCS, and more.

The role emphasizes IaC with Terraform, Airflow orchestration, and Kubernetes, enabling batch and real‑time processing. Strong Python, SQL, and cloud data expertise are essential.