Senior Data Engineer (PySpark, Kafka, GCP)

Apex Systems

Mexico

On-site

PHP 5,538,461 - 7,999,999

Full time

14 days+

Get more replies from employers

Send a job-specific resume in minutes.

Benefits offered by this job

100% payroll

30-day Christmas bonus

15-day vacations + 5 floating days (70

Job summary

Apex Systems is seeking an experienced data/ software engineer to design and optimize data pipelines. You will work with PySpark, Spark SQL, and Spark Streaming to move and transform large real-time and batch data, enabling reliable analytics and scalable data architectures.

The role requires 5+ years in data or software engineering, strong SQL skills, and hands-on experience with GCP, Kafka, and CI/CD pipelines. Join a team delivering impactful data solutions in a dynamic IT environment.

Qualifications

Bachelor’s degree in Computer Science, Engineering, Information Systems, or a related field (or equivalent experience).
5+ years of experience in Data Engineering or Software Engineering.
Strong hands-on experience with Python.
Proven experience building and optimizing batch and real-time data pipelines using PySpark, Spark SQL, and Spark Streaming.
Strong expertise writing and optimizing complex SQL queries for large-scale datasets.
Experience working with Apache Kafka and event-driven architectures.
Experience with Google Cloud Platform (GCP), including BigQuery and Dataproc.
Experience working with RESTful APIs for data ingestion and integration.
Hands-on experience implementing CI/CD pipelines using GitLab CI.
Strong understanding of data modeling, distributed systems, and scalable data architectures.
Experience working with relational and NoSQL databases.
Experience supporting high-volume, real-time data processing environments.

Skills

Python

Batch & real-time pipelines

SQL

Data modeling

Distributed systems

RESTful APIs

CI/CD

GCP

Kafka

Relational & NoSQL databases

Education

Bachelor's degree in Computer Science/Engineering/IS

Tools

GitLab CI

Apache Spark (PySpark, Spark SQL, Spark Streaming)

Kafka

BigQuery

Dataproc

RESTful APIs

Job description

Who we are

Apex Systems is a leading Data and Digital Transformation professional services organisation focused on delivering solutions that deliver real business value. We build authentic partnerships with our clients, offering objective counsel from concept to deployment to ensure a consistent voice in a dynamic IT environment

Required Experience & Qualifications

Bachelor’s degree in Computer Science, Engineering, Information Systems, or a related field (or equivalent experience).
5+ years of experience in Data Engineering or Software Engineering.
Strong hands-on experience with Python.
Proven experience building and optimizing batch and real-time data pipelines using PySpark, Spark SQL, and Spark Streaming.
Strong expertise writing and optimizing complex SQL queries for large-scale datasets.
Experience working with Apache Kafka and event-driven architectures.
Experience with Google Cloud Platform (GCP), including BigQuery and Dataproc.
Experience working with RESTful APIs for data ingestion and integration.
Hands-on experience implementing CI/CD pipelines using GitLab CI.
Strong understanding of data modeling, distributed systems, and scalable data architectures.
Experience working with relational and NoSQL databases.
Experience supporting high-volume, real-time data processing environments.

Preferred Qualifications

Experience with Scala.
Exposure to Java and Spring Boot development.
Familiarity with pair programming, code reviews, and collaborative engineering practices.
Experience designing and supporting event-driven data platforms.
Strong communication skills and ability to explain technical decisions and trade-offs.
Experience partnering with cross-functional teams in agile environments.

Technical Stack Core