Azure Databricks Data Engineering Lead

Publicis Groupe Holdings B.V; Bogotá ciudad; Presencial; COP 90.000.000 - 140.000.000; Jornada completa

Descripción de la vacante

Publicis Sapient, part of Publicis Groupe, seeks a Big Data Manager / Data Engineering Lead to design and deliver enterprise-scale, cloud-native data solutions on Microsoft Azure. You will lead a 3-4 person Data Engineer team, drive architecture, code reviews, and delivery while remaining hands-on with Azure Databricks, ADLS, Spark, Scala, and SQL.

The role emphasizes end-to-end data pipelines, data quality, performance optimization, and collaboration with architects, QA, and client stakeholders

Formación

• 8+ years of hands-on experience in Data Engineering, Big Data Engineering, Data Warehousing development, or related data-platform engineering.
• Strong hands-on expertise with Azure Databricks, Apache Spark, ADLS, Scala, and SQL, including Spark performance optimization and SQL transformation/tuning.
• Proven experience designing and implementing enterprise-grade ETL/ELT pipelines and reusable ingestion/transformation patterns.
• Strong understanding of batch and streaming architectures, including Spark Structured Streaming.
• Demonstrated experience leading and mentoring a small engineering team (3-4 Data Engineers).
• Experience translating business requirements into solution designs with focus on scalability and data quality.
• Experience in Agile delivery models and collaborating across multiple stakeholder groups.
• Knowledge of CI/CD, testing, logging/monitoring, deployment, and production-support practices.

Responsabilidades

• Design scalable Azure cloud-native data solutions and define technical approaches for enterprise batch and streaming data pipelines.
• Lead a team of 3-4 Data Engineers, providing technical direction, mentoring, code reviews, and delivery coordination.
• Build and standardize reusable data engineering components for batch and streaming ingestion, transformation, and data movement across layers.
• Implement reliable Databricks and ADLS pipelines with attention to performance, scalability, reliability, and monitoring.
• Collaborate with architects, analysts, QA, client stakeholders, and other teams to translate requirements into designs.
• Debug and optimize pipelines and codebases, pursuing automation and cost efficiency.
• Drive engineering delivery from backlog planning to production handoff and troubleshooting.

Conocimientos

Azure Databricks

Apache Spark

ADLS

Scala

SQL

Python

PySpark

MongoDB

CI/CD

Observability

Herramientas

Azure Databricks

ADLS

Apache Spark

Scala

SQL

Python

MongoDB

Descripción del empleo

Publicis Sapient, part of Publicis Groupe, seeks a Big Data Manager / Data Engineering Lead to design and deliver enterprise-scale, cloud-native data solutions on Microsoft Azure. You will lead a 3-4 person Data Engineer team, drive architecture, code reviews, and delivery while remaining hands-on with Azure Databricks, ADLS, Spark, Scala, and SQL.

The role emphasizes end-to-end data pipelines, data quality, performance optimization, and collaboration with architects, QA, and client stakeholders