Senior Big Data Engineer (GCP / Python / PySpark /Streaming)

Apex Systems; México; On-site; MXN 900,000 - 1,300,000; Full time

Job summary

Apex Systems is seeking a Senior Data Engineer in Mexico to design and optimize large-scale data pipelines. You will leverage Python, PySpark, and Spark SQL to build robust data workflows, implement real-time processing with Spark Streaming, and integrate with BigQuery on Google Cloud Platform.

Strong SQL and data modeling are essential for performance and quality. Experience with Apache Airflow, Kafka, and CI/CD (GitLab CI) is required, plus the ability to explain design decisions and

Qualifications

• 6+ years of experience in Data Engineering, Big Data Engineering, or related disciplines.

Skills

Python

Big Data

Data Engineering

SQL

NoSQL

Spark

Spark SQL

Scala

RESTful APIs

CI/CD

GCP

Apache Kafka

Spark Streaming

Communication

Education

Bachelor's degree in Computer Science, Engineering, Information Technology, or equivalent

Tools

Apache Airflow

PySpark

Spark SQL

Scala

SQL databases / NoSQL databases

Apache Kafka

Spark Streaming

BigQuery

GitLab CI

Job description

• Bachelor's degree in Computer Science, Engineering, Information Technology, or equivalent practical experience.
• 6+ years of experience in Data Engineering, Big Data Engineering, or related disciplines.
• Strong experience with Python for data engineering and pipeline development.
• Hands-on experience with Apache Airflow for workflow orchestration (required).
• Advanced experience with PySpark, Spark SQL, and distributed data processing.
• Experience with Scala in data engineering environments.
• Strong knowledge of relational and NoSQL database technologies.
• Experience writing and optimizing complex SQL queries.
• Hands-on experience with Apache Kafka and event-driven architectures.
• Experience building and supporting real-time data pipelines using Spark Streaming.
• Strong experience with Google Cloud Platform (GCP), specifically:
• BigQuery
• Experience integrating RESTful APIs into data pipelines and data workflows.
• Experience implementing CI/CD practices using GitLab CI.
• Strong understanding of data modeling, data quality, scalability, and performance optimization.
• Excellent verbal communication and technical problem-solving skills.
• Ability to explain design decisions and implementation approaches in a collaborative setting.