Manager Data Engineer (Azure)

Publicis Groupe ANZ; Bogotá ciudad; Presencial; COP 150.000.000 - 250.000.000; Jornada completa

Descripción de la vacante

Publicis Sapient in Bogotá, Colombia, is seeking a Big Data Manager / Data Engineering Lead with 8+ years of experience to design and deliver enterprise-scale, cloud-native data solutions on Microsoft Azure.

This hands-on leadership role focuses on Azure Databricks, ADLS, Apache Spark, Scala, SQL, and streaming technologies, while guiding a team of 3-4 Data Engineers and coordinating with architects, QA, and client stakeholders to translate requirements into robust data platforms.

Formación

• 8+ years of hands-on experience in Data Engineering, Big Data Engineering, Data Warehousing development, or related data-platform engineering.
• Strong hands-on expertise with Azure Databricks, Apache Spark, ADLS, Scala, and SQL.
• Proven experience designing and implementing enterprise-grade ETL/ELT pipelines.
• Strong understanding of batch and streaming architectures.
• Demonstrated experience leading and mentoring a small engineering team (3-4 Data Engineers).
• Experience translating requirements into scalable solution designs.
• Experience in Agile delivery models and CI/CD practices for data platforms.

Responsabilidades

• Design scalable Azure cloud-native data solutions and define technical approaches for batch and streaming pipelines using Azure Databricks, ADLS, Spark, Scala, and SQL.
• Lead a team of 3-4 Data Engineers, providing direction, mentoring, code reviews, and delivery coordination.
• Build and standardize reusable data engineering components for batch and streaming ingestion, transformation, data quality, and data movement across layers.
• Implement reliable Databricks and ADLS pipelines addressing performance, scalability, monitoring, and non-functional requirements.
• Collaborate with architects, analysts, QA, client stakeholders, and downstream teams to translate business needs into designs.
• Debug and optimize pipelines and codebases, drive automation, performance improvements, and cost efficiency.
• Drive engineering delivery from backlog planning through QA/UAT, production handoff, and troubleshooting.

Conocimientos

Azure Databricks

Apache Spark

ADLS

Scala

SQL

Python / PySpark

Team leadership

Descripción del empleo

Company description

Publicis Sapient is a digital transformation partner helping established organizations get to their future, digitally enabled state, both in the way they work and the way they serve their customers. We help unlock value through a start-up mindset and modern methods, fusing strategy, consulting, and customer experience with agile engineering and problem-solving creativity. United by our core values and our purpose of helping people thrive in the brave pursuit of next, our 20,000+ people in 53 offices around the world combine experience across technology, data sciences, consulting, and customer obsession to accelerate our clients’ businesses through designing the products and services their customers truly value.

Overview

We are looking for a Big Data Manager / Data Engineering Lead with 8+ years of experience in Data Engineering, Big Data, or Data Warehousing to lead the design and delivery of enterprise-scale, cloud-native data solutions on Microsoft Azure.

This is a hands-on technical leadership role focused on Azure Databricks, Azure Data Lake Storage (ADLS), Apache Spark, Scala, SQL, and streaming technologies. You will lead a team of 3-4 Data Engineers, providing technical direction, design guidance, code reviews, mentoring, and delivery oversight while remaining close to solution architecture and engineering.

Responsibilities

Your Impact

• Design scalable Azure cloud-native data solutions and define technical approaches for enterprise batch and streaming data pipelines using Azure Databricks, ADLS, Apache Spark, Scala, and SQL.
• Lead a team of 3-4 Data Engineers, providing technical direction, mentoring, code reviews, design guidance, delivery coordination, and support removing technical blockers.
• Build and standardize reusable data engineering components and patterns supporting batch and streaming ingestion, transformation, data quality, and movement across raw, curated, and consumption layers.
• Implement reliable and maintainable Databricks and ADLS pipelines while addressing performance, scalability, resilience, logging, monitoring, testing, and other non-functional requirements.
• Collaborate with architects, analysts, QA, client stakeholders, and upstream/downstream application teams to translate business and technical requirements into solution and implementation designs.
• Debug and optimize existing pipelines and codebases, identifying opportunities for automation, performance improvement, scalability, reliability, operational efficiency, and reduced run cost.
• Drive engineering delivery from backlog planning and estimation through QA/UAT, deployment readiness, production implementation, operational handoff, and troubleshooting.

Qualifications

Skills & Experience

• 8+ years of hands-on experience in Data Engineering, Big Data Engineering, Data Warehousing development, or related data-platform engineering, including delivery of production data solutions.
• Strong hands-on expertise with Azure Databricks, Apache Spark, ADLS, Scala, and SQL, including Spark performance optimization and SQL transformation/tuning.
• Proven experience designing and implementing enterprise-grade ETL/ELT pipelines, including reusable ingestion and data transformation patterns.
• Strong understanding of batch and streaming architectures, including Spark Structured Streaming or comparable streaming implementation patterns.
• Demonstrated experience leading and mentoring a small engineering team, with the ability to manage 3-4 Data Engineers, guide technical decisions, review code/designs, unblock engineers, and maintain delivery accountability.
• Experience translating business and technical requirements into solution designs and technical designs, including appropriate consideration of scalability, reliability, performance, maintainability, and data quality.
• Experience working within Agile delivery models and collaborating across engineering, architecture, data, QA, and client/business stakeholder groups.
• Working knowledge of CI/CD, testing, source control, observability, logging/monitoring, deployment, and production-support practices for modern data platforms.

Set Yourself Apart With

• Python / PySpark development experience.
• Broader experience across the Microsoft Azure data ecosystem.
• Experience with MongoDB or other NoSQL technologies.
• Experience designing or implementing data governance and data quality frameworks.
• Experience in the Healthcare domain, particularly working with claims, clinical, eligibility, provider, or member data and HIPAA-aware data handling.
• Experience communicating complex technical decisions and trade-offs to both technical and non-technical client stakeholders.