Data Engineer – Spark Pipelines & Databricks | Flexible

NielsenIQ; México; Híbrido; MXN 600.000 - 1.000.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Flexible working environment

Volunteer time off

Descripción de la vacante

NielsenIQ is hiring a Data Engineer to join a team developing and optimizing large-scale data pipelines. You will work with Spark, Python/Scala, Databricks, Airflow, SQL, Docker, and Kubernetes to transform data into actionable client insights.

The role emphasizes building scalable ETL infrastructures, integrating models into pipelines, and delivering well-documented features for support teams. Collaboration and cloud familiarity are important in this hybrid environment.

Formación

• 5–8 years of experience as a Data Engineer.
• Experience modeling or manipulating large data sets is essential.
• Hands-on data engineering experience with Spark, Python/Scala, SQL.
• Direct experience building data pipelines with Apache Spark (Databricks/Fabric) and Airflow.

Responsabilidades

• Maintain and assemble data pipelines for large datasets.
• Perform complex data transformations to generate client insights.
• Integrate finished models into pipelines and build client insights.
• Optimize Spark code for performance and memory usage.
• Build infrastructure for ETL from diverse data sources using Azure, Databricks, and SQL.
• Write and maintain feature documentation and hand off to L2 support.

Conocimientos

Spark (PySpark/Scala)

Python

SQL

Data modeling

Herramientas

Databricks

Airflow

Docker

Kubernetes

Azure

Descripción del empleo

NielsenIQ is hiring a Data Engineer to join a team developing and optimizing large-scale data pipelines. You will work with Spark, Python/Scala, Databricks, Airflow, SQL, Docker, and Kubernetes to transform data into actionable client insights.

The role emphasizes building scalable ETL infrastructures, integrating models into pipelines, and delivering well-documented features for support teams. Collaboration and cloud familiarity are important in this hybrid environment.