Data Engineer: Spark & Databricks Pipelines

NielsenIQ; Ciudad de México; Presencial; MXN 600.000 - 900.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Flexible working environment

Volunteer time off

LinkedIn Learning

Employee-Assistance-Program (EAP)

Descripción de la vacante

NIQ is seeking a Data Engineer in Mexico City to design and maintain large-scale data pipelines using Spark, PySpark, and Databricks. You will optimize Spark jobs and help integrate models into production pipelines to generate client insights.

The role requires 5–8 years of hands-on data engineering experience, strong Python/Scala and SQL skills, and knowledge of data warehousing. Collaboration and documentation are emphasized.

Formación

• 5–8 years of experience as a Data Engineer.
• Experience modeling or manipulating large data sets.
• Hands-on data engineer experience with Spark (Python/Scala).
• Knowledge of data warehousing concepts & methodologies.
• Programming in Python/Scala, SQL.
• Direct experience building data pipelines using Apache Spark (Databricks/Fabric) and Airflow.

Responsabilidades

• Maintain and assemble data pipelines that process large datasets and perform complex transformations.
• Integrate finished models into pipelines to build client insights.
• Optimize Spark code for performance and memory efficiency.
• Build infrastructure for ETL from various sources using Azure, Databricks and SQL technologies.
• Write and maintain documentation for features and transition them to L2 support.

Conocimientos

Data engineering

Python

Scala

Spark

SQL

Big data

CI/CD

Herramientas

Databricks

Airflow

Docker

Kubernetes

Azure

Descripción del empleo

NIQ is seeking a Data Engineer in Mexico City to design and maintain large-scale data pipelines using Spark, PySpark, and Databricks. You will optimize Spark jobs and help integrate models into production pipelines to generate client insights.

The role requires 5–8 years of hands-on data engineering experience, strong Python/Scala and SQL skills, and knowledge of data warehousing. Collaboration and documentation are emphasized.