Senior Data Engineer - PySpark, Kafka & Real-Time Pipelines

Apex Systems; Mexico; On-site; PHP 5,538,461 - 7,999,999; Full time

Benefits offered by this job

100% payroll

30-day Christmas bonus

15-day vacations + 5 floating days (70

Job summary

Apex Systems is seeking an experienced data/ software engineer to design and optimize data pipelines. You will work with PySpark, Spark SQL, and Spark Streaming to move and transform large real-time and batch data, enabling reliable analytics and scalable data architectures.

The role requires 5+ years in data or software engineering, strong SQL skills, and hands-on experience with GCP, Kafka, and CI/CD pipelines. Join a team delivering impactful data solutions in a dynamic IT environment.

Qualifications

• Bachelor’s degree in Computer Science, Engineering, Information Systems, or a related field (or equivalent experience).
• 5+ years of experience in Data Engineering or Software Engineering.
• Strong hands-on experience with Python.
• Proven experience building and optimizing batch and real-time data pipelines using PySpark, Spark SQL, and Spark Streaming.
• Strong expertise writing and optimizing complex SQL queries for large-scale datasets.
• Experience working with Apache Kafka and event-driven architectures.
• Experience with Google Cloud Platform (GCP), including BigQuery and Dataproc.
• Experience working with RESTful APIs for data ingestion and integration.
• Hands-on experience implementing CI/CD pipelines using GitLab CI.
• Strong understanding of data modeling, distributed systems, and scalable data architectures.
• Experience working with relational and NoSQL databases.
• Experience supporting high-volume, real-time data processing environments.

Skills

Python

Batch & real-time pipelines

SQL

Data modeling

Distributed systems

RESTful APIs

CI/CD

GCP

Kafka

Relational & NoSQL databases

Education

Bachelor's degree in Computer Science/Engineering/IS

Tools

GitLab CI

Apache Spark (PySpark, Spark SQL, Spark Streaming)

Kafka

BigQuery

Dataproc

RESTful APIs

Job description

Apex Systems is seeking an experienced data/ software engineer to design and optimize data pipelines. You will work with PySpark, Spark SQL, and Spark Streaming to move and transform large real-time and batch data, enabling reliable analytics and scalable data architectures.

The role requires 5+ years in data or software engineering, strong SQL skills, and hands-on experience with GCP, Kafka, and CI/CD pipelines. Join a team delivering impactful data solutions in a dynamic IT environment.