Lead Data Engineer - Databricks & Data Ingestion

Pyramid Consulting, Inc

Aguascalientes

Hybrid

MXN 1,200,000 - 1,800,000

Full time

5 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Pyramid Consulting, Inc. is seeking an experienced Lead Data Engineer in Aguascalientes, Mexico, hybrid work. The role focuses on building scalable data pipelines using Databricks, Lakeflow Connect, Delta Lake, and Spark to ingest and process enterprise data from multiple sources.

The ideal candidate will lead architecture, optimize ETL/ELT workflows, ensure data quality and platform reliability, and mentor junior engineers while collaborating with product, analytics, and cloud teams.

Qualifications

  • 8+ years in data engineering or big data technologies.
  • Hands-on experience with Databricks and enterprise-scale data ingestion solutions.
  • Proven experience designing and implementing scalable data pipelines and ETL/ELT frameworks.
  • Experience leading technical teams, mentoring engineers, and driving architecture decisions.
  • Strong communication, problem-solving, and stakeholder collaboration skills.

Responsibilities

  • Design and implement scalable data ingestion frameworks using Databricks Native Data Loader, Lakeflow Connect, and other Databricks-native capabilities.
  • Develop and optimize batch and streaming data pipelines integrating Oracle, MySQL, SQL Server, APIs, and other enterprise data sources.
  • Build robust ETL/ELT pipelines using Databricks, Apache Spark, Delta Lake, and cloud-native technologies.
  • Architect end-to-end data ingestion, transformation, storage, validation, and monitoring solutions.
  • Design and maintain scalable data pipeline architectures to transform raw and unstructured data into analytics-ready datasets.
  • Implement data quality checks, validation frameworks, pipeline monitoring, and error-handling mechanisms.
  • Perform performance tuning, root cause analysis, and optimization of data pipelines and distributed processing workloads.
  • Drive automation, infrastructure optimization, scalability, and continuous improvement initiatives.
  • Implement unit testing, integration testing, and Test-Driven Development (TDD) practices.
  • Ensure data governance, security, reliability, and compliance with enterprise engineering standards.
  • Provide technical leadership, mentorship, and guidance to junior data engineers.
  • Collaborate with Product, Architecture, Analytics, Engineering, and Business teams to translate requirements into technical solutions.
  • Drive engineering best practices across coding standards, CI/CD, DevOps, documentation, and code reviews.
  • Participate in architecture discussions, technical design reviews, Agile ceremonies, sprint planning, and estimation.

Skills

Databricks
Lakeflow Connect
Delta Lake
Apache Spark
Data Ingestion
ETL/ELT
Cloud Data Platforms
CI/CD & DevOps
TDD & Testing
Data Governance

Tools

Databricks Native Data Loader
Lakeflow Connect
Delta Lake
Apache Spark

Job description

Pyramid Consulting, Inc. is seeking an experienced Lead Data Engineer in Aguascalientes, Mexico, hybrid work. The role focuses on building scalable data pipelines using Databricks, Lakeflow Connect, Delta Lake, and Spark to ingest and process enterprise data from multiple sources.

The ideal candidate will lead architecture, optimize ETL/ELT workflows, ensure data quality and platform reliability, and mentor junior engineers while collaborating with product, analytics, and cloud teams.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer
Data Engineer

Pyramid Consulting, Inc • Aguascalientes

Hybrid
MXN 1,200,000 - 1,800,000
Remote Data Engineer (Mexico) - Databricks & AI/ML Pipelines
Remote Data Engineer (Mexico) - Databricks & AI/ML Pipelines

NEPSE Trading • Mexico

Remote
MXN 420,000 - 780,000
Health insurance
Flexible schedule and remote working
Employee rewards program and paid Par‑
Lead Data Engineer: Scalable Pipelines & Lakehouse
Lead Data Engineer: Scalable Pipelines & Lakehouse

EPAM Systems Inc • Mexico

Remote
MXN 900,000 - 1,300,000
Healthcare benefits
Global career opportunities
Learning & development programs
+1
Lead Data Engineer / Databricks
Lead Data Engineer / Databricks

Capgemini • Aguascalientes

On-site
MXN 900,000 - 1,300,000
Databricks Data Engineer: Build Scalable Pipelines
Databricks Data Engineer: Build Scalable Pipelines

Accenture PLC • Nuevo León

On-site
MXN 400,000 - 900,000
Career development according to your f
Access to innovative methodology and工具
Direct contact with experts worldwide
+4
Databricks Data Engineer: Build Pipelines, Lakehouse
Databricks Data Engineer: Build Pipelines, Lakehouse

Zurich • Estado de México

Hybrid
MXN 1,032,346 - 1,290,433
Senior Data Engineer - Spark & Databricks, Global Projects
Senior Data Engineer - Spark & Databricks, Global Projects

EPAM Systems Inc • Mexico

On-site
MXN 600,000 - 900,000
Healthcare benefits
Paid time off and sick leave
Upskilling and certification courses
+1
Data Engineer Sr
Data Engineer Sr

Turtle Trax S.A. • Región Centro

Hybrid
MXN 800,000 - 1,100,000
Data Engineer - Databricks
Data Engineer - Databricks

Zurich Insurance • Región Centro

On-site
MXN 860,289 - 1,204,404
Data Engineer: dbt & Snowflake Pipelines (Remote)
Data Engineer: dbt & Snowflake Pipelines (Remote)

AgileEngine, LLC. • Región Centro

On-site
MXN 480,000 - 640,000
Professional growth
Competitive compensation (USD)
Exciting projects
+1