Senior Big Data Engineer: Spark & Airflow Pipelines

Apex Systems; México; On-site; MXN 900,000 - 1,300,000; Full time

Job summary

Apex Systems is seeking a Senior Data Engineer in Mexico to design and optimize large-scale data pipelines. You will leverage Python, PySpark, and Spark SQL to build robust data workflows, implement real-time processing with Spark Streaming, and integrate with BigQuery on Google Cloud Platform.

Strong SQL and data modeling are essential for performance and quality. Experience with Apache Airflow, Kafka, and CI/CD (GitLab CI) is required, plus the ability to explain design decisions and

Qualifications

• 6+ years of experience in Data Engineering, Big Data Engineering, or related disciplines.

Skills

Python

Big Data

Data Engineering

SQL

NoSQL

Spark

Spark SQL

Scala

RESTful APIs

CI/CD

GCP

Apache Kafka

Spark Streaming

Communication

Education

Bachelor's degree in Computer Science, Engineering, Information Technology, or equivalent

Tools

Apache Airflow

PySpark

Spark SQL

Scala

SQL databases / NoSQL databases

Apache Kafka

Spark Streaming

BigQuery

GitLab CI

Job description

Apex Systems is seeking a Senior Data Engineer in Mexico to design and optimize large-scale data pipelines. You will leverage Python, PySpark, and Spark SQL to build robust data workflows, implement real-time processing with Spark Streaming, and integrate with BigQuery on Google Cloud Platform.

Strong SQL and data modeling are essential for performance and quality. Experience with Apache Airflow, Kafka, and CI/CD (GitLab CI) is required, plus the ability to explain design decisions and