Manager Data Engineer (Azure)

Publicis Groupe Holdings B.V; Bogotá ciudad; Presencial; COP 90.000.000 - 140.000.000; Jornada completa

Descripción de la vacante

Publicis Sapient, part of Publicis Groupe, seeks a Big Data Manager / Data Engineering Lead to design and deliver enterprise-scale, cloud-native data solutions on Microsoft Azure. You will lead a 3-4 person Data Engineer team, drive architecture, code reviews, and delivery while remaining hands-on with Azure Databricks, ADLS, Spark, Scala, and SQL.

The role emphasizes end-to-end data pipelines, data quality, performance optimization, and collaboration with architects, QA, and client stakeholders

Formación

• 8+ years of hands-on experience in Data Engineering, Big Data Engineering, Data Warehousing development, or related data-platform engineering.
• Strong hands-on expertise with Azure Databricks, Apache Spark, ADLS, Scala, and SQL, including Spark performance optimization and SQL transformation/tuning.
• Proven experience designing and implementing enterprise-grade ETL/ELT pipelines and reusable ingestion/transformation patterns.
• Strong understanding of batch and streaming architectures, including Spark Structured Streaming.
• Demonstrated experience leading and mentoring a small engineering team (3-4 Data Engineers).
• Experience translating business requirements into solution designs with focus on scalability and data quality.
• Experience in Agile delivery models and collaborating across multiple stakeholder groups.
• Knowledge of CI/CD, testing, logging/monitoring, deployment, and production-support practices.

Responsabilidades

• Design scalable Azure cloud-native data solutions and define technical approaches for enterprise batch and streaming data pipelines.
• Lead a team of 3-4 Data Engineers, providing technical direction, mentoring, code reviews, and delivery coordination.
• Build and standardize reusable data engineering components for batch and streaming ingestion, transformation, and data movement across layers.
• Implement reliable Databricks and ADLS pipelines with attention to performance, scalability, reliability, and monitoring.
• Collaborate with architects, analysts, QA, client stakeholders, and other teams to translate requirements into designs.
• Debug and optimize pipelines and codebases, pursuing automation and cost efficiency.
• Drive engineering delivery from backlog planning to production handoff and troubleshooting.

Conocimientos

Azure Databricks

Apache Spark

ADLS

Scala

SQL

Python

PySpark

MongoDB

CI/CD

Observability

Herramientas

Azure Databricks

ADLS

Apache Spark

Scala

SQL

Python

MongoDB

Descripción del empleo

Company description

Publicis Sapient is a digital transformation partner helping established organizations get to their future, digitally enabled state, both in the way they work and the way they serve their customers. We help unlock value through a start-up mindset and modern methods, fusing strategy, consulting, and customer experience with agile engineering and problem-solving creativity. United by our core values and our purpose of helping people thrive in the brave pursuit of next, our 20,000+ people in 53 offices around the world combine experience across technology, data sciences, consulting, and customer obsession to accelerate our clients’ businesses through designing the products and services their customers truly value.

Overview

We are looking for a Big Data Manager / Data Engineering Lead with 8+ years of experience in Data Engineering, Big Data, or Data Warehousing to lead the design and delivery of enterprise-scale, cloud-native data solutions on Microsoft Azure.

This is a hands-on technical leadership role focused on Azure Databricks, Azure Data Lake Storage (ADLS), Apache Spark, Scala, SQL, and streaming technologies. You will lead a team of 3-4 Data Engineers, providing technical direction, design guidance, code reviews, mentoring, and delivery oversight while remaining close to solution architecture and engineering.

Responsibilities

Your Impact

• Design scalable Azure cloud-native data solutions and define technical approaches for enterprise batch and streaming data pipelines using Azure Databricks, ADLS, Apache Spark, Scala, and SQL.
• Lead a team of 3-4 Data Engineers, providing technical direction, mentoring, code reviews, design guidance, delivery coordination, and support removing technical blockers.
• Build and standardize reusable data engineering components and patterns supporting batch and streaming ingestion, transformation, data quality, and movement across raw, curated, and consumption layers.
• Implement reliable and maintainable Databricks and ADLS pipelines while addressing performance, scalability, resilience, logging, monitoring, testing, and other non-functional requirements.
• Collaborate with architects, analysts, QA, client stakeholders, and upstream/downstream application teams to translate business and technical requirements into solution and implementation designs.
• Debug and optimize existing pipelines and codebases, identifying opportunities for automation, performance improvement, scalability, reliability, operational efficiency, and reduced run cost.
• Drive engineering delivery from backlog planning and estimation through QA/UAT, deployment readiness, production implementation, operational handoff, and troubleshooting.

Qualifications

Skills & Experience

• 8+ years of hands-on experience in Data Engineering, Big Data Engineering, Data Warehousing development, or related data-platform engineering, including delivery of production data solutions.
• Strong hands-on expertise with Azure Databricks, Apache Spark, ADLS, Scala, and SQL, including Spark performance optimization and SQL transformation/tuning.
• Proven experience designing and implementing enterprise-grade ETL/ELT pipelines, including reusable ingestion and data transformation patterns.
• Strong understanding of batch and streaming architectures, including Spark Structured Streaming or comparable streaming implementation patterns.
• Demonstrated experience leading and mentoring a small engineering team, with the ability to manage 3-4 Data Engineers, guide technical decisions, review code/designs, unblock engineers, and maintain delivery accountability.
• Experience translating business and technical requirements into solution designs and technical designs, including appropriate consideration of scalability, reliability, performance, maintainability, and data quality.
• Experience working within Agile delivery models and collaborating across engineering, architecture, data, QA, and client/business stakeholder groups.
• Working knowledge of CI/CD, testing, source control, observability, logging/monitoring, deployment, and production-support practices for modern data platforms.

Set Yourself Apart With

• Python / PySpark development experience.
• Broader experience across the Microsoft Azure data ecosystem.
• Experience with MongoDB or other NoSQL technologies.
• Experience designing or implementing data governance and data quality frameworks.
• Experience in the Healthcare domain, particularly working with claims, clinical, eligibility, provider, or member data and HIPAA-aware data handling.
• Experience communicating complex technical decisions and trade-offs to both technical and non-technical client stakeholders.