Spark Software Engineer - AI/LLMs & Data Pipelines

Hitachi, Ltd.; Región Centro; Presencial; MXN 1.004.000 - 1.562.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Exciting Projects

Collaborative Environment

Work-Life Balance

Professional Development

Excellent Benefits

Descripción de la vacante

GlobalLogic is seeking a skilled Data Engineer to join our team in Mexico. You will develop and maintain tools to upgrade Apache Spark versions, build robust data validation pipelines, and write clean, scalable code for data processing utilities.

You will explore AI models to aid migrations, work with large-scale data systems, and collaborate with a global team to ensure data parity, performance, and quality across projects.

Formación

• Experience: 4–8 years in professional software development.
• Strong proficiency in Java and/or Python.
• Hands-on experience with Apache Spark and Hadoop ecosystem.
• Exposure to AI tools, methodologies, and LLMs, esp. Gemini models/APIs.
• Proven experience in testing, debugging, and maintaining batch or streaming data pipelines.
• Nice to have: Cloud environments like Google Cloud Platform / Dataproc.
• Familiarity with unit testing frameworks and CI/CD pipelines for data engineering.
• Experience with automated data diffing or validation tools.

Responsabilidades

• Develop and maintain automated tools and scripts to facilitate and accelerate Spark version upgrades.
• Design and execute data validation pipelines and data quality checks post-upgrade to ensure parity, integrity, and performance.
• Write clean, scalable, maintainable code for data processing or toolchain utilities.
• Explore and integrate AI models and methodologies to assist in code assessment, error detection, or translation tasks during migrations.

Conocimientos

Java

Python

Apache Spark

Hadoop ecosystem

Google Gemini

Data pipelines testing

GCP / Dataproc

Unit testing frameworks

CI/CD pipelines

Automated data validation

Herramientas

CI/CD tools

Descripción del empleo

GlobalLogic is seeking a skilled Data Engineer to join our team in Mexico. You will develop and maintain tools to upgrade Apache Spark versions, build robust data validation pipelines, and write clean, scalable code for data processing utilities.

You will explore AI models to aid migrations, work with large-scale data systems, and collaborate with a global team to ensure data parity, performance, and quality across projects.