Data Engineer

Pyramid Consulting, Inc

Aguascalientes

Híbrido

MXN 1.200.000 - 1.800.000

Jornada completa

Hace 5 días
Sé de los primeros/as/es en solicitar esta vacante

Consigue una respuesta de este empleador

Tu currículum y tu carta de presentación, adaptados exactamente a lo que buscan contratar.

Supera los filtros ATS

Descripción de la vacante

Pyramid Consulting, Inc. is seeking an experienced Lead Data Engineer in Aguascalientes, Mexico, hybrid work. The role focuses on building scalable data pipelines using Databricks, Lakeflow Connect, Delta Lake, and Spark to ingest and process enterprise data from multiple sources.

The ideal candidate will lead architecture, optimize ETL/ELT workflows, ensure data quality and platform reliability, and mentor junior engineers while collaborating with product, analytics, and cloud teams.

Formación

  • 8+ years in data engineering or big data technologies.
  • Hands-on experience with Databricks and enterprise-scale data ingestion solutions.
  • Proven experience designing and implementing scalable data pipelines and ETL/ELT frameworks.
  • Experience leading technical teams, mentoring engineers, and driving architecture decisions.
  • Strong communication, problem-solving, and stakeholder collaboration skills.

Responsabilidades

  • Design and implement scalable data ingestion frameworks using Databricks Native Data Loader, Lakeflow Connect, and other Databricks-native capabilities.
  • Develop and optimize batch and streaming data pipelines integrating Oracle, MySQL, SQL Server, APIs, and other enterprise data sources.
  • Build robust ETL/ELT pipelines using Databricks, Apache Spark, Delta Lake, and cloud-native technologies.
  • Architect end-to-end data ingestion, transformation, storage, validation, and monitoring solutions.
  • Design and maintain scalable data pipeline architectures to transform raw and unstructured data into analytics-ready datasets.
  • Implement data quality checks, validation frameworks, pipeline monitoring, and error-handling mechanisms.
  • Perform performance tuning, root cause analysis, and optimization of data pipelines and distributed processing workloads.
  • Drive automation, infrastructure optimization, scalability, and continuous improvement initiatives.
  • Implement unit testing, integration testing, and Test-Driven Development (TDD) practices.
  • Ensure data governance, security, reliability, and compliance with enterprise engineering standards.
  • Provide technical leadership, mentorship, and guidance to junior data engineers.
  • Collaborate with Product, Architecture, Analytics, Engineering, and Business teams to translate requirements into technical solutions.
  • Drive engineering best practices across coding standards, CI/CD, DevOps, documentation, and code reviews.
  • Participate in architecture discussions, technical design reviews, Agile ceremonies, sprint planning, and estimation.

Conocimientos

Databricks
Lakeflow Connect
Delta Lake
Apache Spark
Data Ingestion
ETL/ELT
Cloud Data Platforms
CI/CD & DevOps
TDD & Testing
Data Governance

Herramientas

Databricks Native Data Loader
Lakeflow Connect
Delta Lake
Apache Spark

Descripción del empleo

We're Hiring: Lead Data Engineer - Databricks (Data Ingestion & Lakeflow Connect)
Role: Lead Data Engineer
Location: Aguascalientes, Mexico (Hybrid)
Specialization:

Databricks, Lakeflow Connect, Data Engineering & Cloud Data Platforms

Job Description

We are looking for an experienced Lead Data Engineer with strong expertise in Databricks, enterprise data ingestion, and large-scale data engineering to lead the design and implementation of scalable, high-performance data pipelines.

The ideal candidate should have hands-on experience with Databricks Native Data Loader, Lakeflow Connect, Delta Lake, Apache Spark, and enterprise data integration from relational databases such as Oracle, MySQL, and SQL Server.

The role involves driving data engineering architecture, optimizing ETL/ELT workflows, ensuring data quality and platform reliability, and providing technical leadership to engineering teams.

Key Responsibilities
  • Design and implement scalable data ingestion frameworks using Databricks Native Data Loader, Lakeflow Connect, and other Databricks-native capabilities.
  • Develop and optimize batch and streaming data pipelines integrating Oracle, MySQL, SQL Server, APIs, and other enterprise data sources.
  • Build robust ETL/ELT pipelines using Databricks, Apache Spark, Delta Lake, and cloud-native technologies.
  • Architect end-to-end data ingestion, transformation, storage, validation, and monitoring solutions.
  • Design and maintain scalable data pipeline architectures to transform raw and unstructured data into analytics-ready datasets.
  • Implement data quality checks, validation frameworks, pipeline monitoring, and error-handling mechanisms.
  • Perform performance tuning, root cause analysis, and optimization of data pipelines and distributed processing workloads.
  • Drive automation, infrastructure optimization, scalability, and continuous improvement initiatives.
  • Implement unit testing, integration testing, and Test-Driven Development (TDD) practices.
  • Ensure data governance, security, reliability, and compliance with enterprise engineering standards.
  • Provide technical leadership, mentorship, and guidance to junior data engineers.
  • Collaborate with Product, Architecture, Analytics, Engineering, and Business teams to translate requirements into technical solutions.
  • Drive engineering best practices across coding standards, CI/CD, DevOps, documentation, and code reviews.
  • Participate in architecture discussions, technical design reviews, Agile ceremonies, sprint planning, and estimation.
Required Technical Skills
  • Databricks: Databricks Native Data Loader, Lakeflow Connect, Delta Lake.
  • Data Engineering: ETL/ELT, Data Ingestion, Data Integration, Data Transformation.
  • Big Data Technologies: Apache Spark, PySpark, Distributed Data Processing.
  • Databases: Oracle, MySQL, Microsoft SQL Server.
  • Data Processing: Batch Processing, Streaming, Data Pipelines.
  • Cloud & Architecture: Cloud Data Platforms, Scalable Data Architecture.
  • Engineering Practices: CI/CD, DevOps, TDD, Unit Testing, Integration Testing.
Experience & Qualifications
  • 8+ years of experience in Data Engineering, Data Integration, or Big Data technologies.
  • Strong hands-on experience with Databricks and enterprise-scale data ingestion solutions.
  • Proven experience designing and implementing scalable data pipelines and ETL/ELT frameworks.
  • Experience leading technical teams, mentoring engineers, and driving architecture decisions.
  • Strong communication, problem-solving, and stakeholder collaboration skills.

#Hiring #LeadDataEngineer #Databricks #LakeflowConnect #DataEngineering #DeltaLake #ApacheSpark #PySpark #DataIngestion #ETL #ELT #BigData #CloudDataEngineering #DataArchitecture

Consigue la evaluación confidencial y gratuita de tu currículum.

o arrastra y suelta tu archivo aquí

Similar jobs

Puestos de trabajo similares que vale la pena comparar

Data Engineer Mexico
Data Engineer Mexico

V4C.ai • México

A distancia
MXN 420.000 - 660.000
Health insurance
Growth opportunities
Parental leave
+1
Lead Data Engineer - Databricks & Data Ingestion
Lead Data Engineer - Databricks & Data Ingestion

Pyramid Consulting, Inc • Aguascalientes

Híbrido
MXN 1.200.000 - 1.800.000
Data Engineer Sr
Data Engineer Sr

Turtle Trax S.A. • Región Centro

Híbrido
MXN 800.000 - 1.100.000
Lead Data Engineer / Databricks
Lead Data Engineer / Databricks

Capgemini • Aguascalientes

Presencial
MXN 900.000 - 1.300.000
Data Engineer - Databricks
Data Engineer - Databricks

Zurich Insurance • Región Centro

Presencial
MXN 860.289 - 1.204.404
Senior Data Engineer
Senior Data Engineer

Code17tek • Estado de México

Presencial
MXN 900.000 - 1.500.000
Data Engineer Mexico
Data Engineer Mexico

NEPSE Trading • México

A distancia
MXN 420.000 - 780.000
Health insurance
Flexible schedule and remote working
Employee rewards program and paid Par‑
Senior Azure Databricks and PySpark
Senior Azure Databricks and PySpark

Daikin Manufacturing México • Monterrey

Híbrido
MXN 900.000 - 1.400.000
Data Solution Architect
Data Solution Architect

Turtle Trax S.A. • Región Centro

A distancia
MXN 900.000 - 1.500.000
Data QA Engineer
Data QA Engineer

KTek Resourcing • Ciudad de México

Presencial
MXN 152.334 - 182.466