Senior Data Engineer: Spark & Cloud ETL Expert

LTM Limited; Mexico; On-site; MXN 480,000 - 960,000; Full time

Job summary

LTM Limited in Mexico is seeking a data engineering professional to design, build and maintain scalable ETL/ELT pipelines using PySpark and Spark SQL. You will work with cloud data infrastructure across AWS, Azure, and Google Cloud, optimizing data warehouses and data lake storage for performance and accessibility.

You will optimize Spark jobs, apply performance tuning, and implement automated workflows with Airflow while collaborating with data scientists and analysts to translate business

Qualifications

• Mandatory Databricks Certified Associate Developer for Apache Spark certification.
• Experience designing and maintaining ETL/ELT data pipelines with PySpark and Spark SQL.
• Strong familiarity with cloud data infrastructure across AWS, Azure and GCP.

Responsibilities

• Design, build and maintain scalable ETL/ELT data pipelines using PySpark and Spark SQL.
• Deploy and manage cloud data infrastructure components on AWS, Azure and GCP.
• Optimize data warehouse storage, partitioning and indexing for performance and accessibility.
• Proactively tune Spark jobs and resolve bottlenecks.
• Ingest high-volume data from structured and unstructured sources into a data ecosystem.
• Implement automated workflows with Airflow or platform-native schedulers.
• Collaborate with data scientists and analysts to translate business requirements into data solutions.

Skills

Data pipeline development

Performance tuning

Cross-functional collaboration

Tools

Databricks

Spark SQL

PySpark

Hive

Airflow

AWS EMR

Azure Databricks

Azure Synapse

GCP (Dataproc/BigQuery)

Job description

LTM Limited in Mexico is seeking a data engineering professional to design, build and maintain scalable ETL/ELT pipelines using PySpark and Spark SQL. You will work with cloud data infrastructure across AWS, Azure, and Google Cloud, optimizing data warehouses and data lake storage for performance and accessibility.

You will optimize Spark jobs, apply performance tuning, and implement automated workflows with Airflow while collaborating with data scientists and analysts to translate business