Senior Data Engineer

Publicis Sapient; Bengaluru; On-site; INR 2,500,000 - 4,000,000; Full time

Job summary

Publicis Sapient is seeking a Senior Data Engineer focused on Azure big data and GenAI solutions. You will design and deploy scalable data architectures, migrate data to cloud-native systems, and integrate GenAI components into data pipelines.

Ideal candidates have 5.5–8 years of cloud data experience, strong Python/PySpark skills, and hands-on Azure expertise (ADF, Databricks, Synapse). Collaboration across teams and strong DevOps practices are essential.

Qualifications

• BS/MS in Computer Science, Mathematics, Engineering, Statistics, or related field; advanced degrees preferred.
• Proficient in Python and distributed computing with PySpark/Apache Spark.
• Deep hands-on architectural experience with Azure cloud services (ADF, Databricks, Synapse/SQL).
• Experience designing and maintaining large-scale enterprise Data Warehouses.
• Strong SQL scripting, window functions, and query optimization.
• Familiarity with DevOps, CI/CD for data applications.
• Experience building/ deploying GenAI models and LLM tooling in production.
• Experience with Azure Key Vault for credential isolation.

Responsibilities

• Lead design, implementation, and deployment of large-scale big data architectures on Azure.
• Drive data migration from on-premises to cloud-native solutions with minimal disruption.
• Incorporate ML and GenAI components (RAG/LLMs) into data pipelines.
• Architect and maintain automated end-to-end data pipelines with ADF and Databricks.
• Analyze source data, perform profiling and schema mapping for data warehouses.
• Embed CI/CD practices into data processing solutions for reliable delivery.
• Collaborate with stakeholders to adapt to new data technologies and resolve bottlenecks.

Skills

Python

PySpark

Apache Spark

Azure Data Factory (ADF)

Azure Databricks

Azure Synapse / SQL Server

SQL

DevOps / CI/CD

GenAI / LLM tooling

Azure Key Vault

Education

BS/MS in Computer Science / Engineering

Tools

Azure Databricks

Azure Synapse

Azure Data Factory

Azure Functions

Azure Logic Apps

Azure WebApps

Job description

At Publicis Sapient, we don't just build technology; we create transformative solutions. By integrating Generative AI, advanced machine learning, and modern cloud data architectures into our ecosystem, we're shaping a new era of intelligent, data-driven digital experiences for global clients. From engineering large-scale automated data pipelines to implementing self-service analytics frameworks and productionizing LLM architectures, we leverage cutting-edge technologies to solve complex business challenges.

Join us in driving innovation and delivering true business value to our customers through the strategic application of advanced big data systems, modern DevOps practices, and next-generation AI.

About the Team

The Data and AI Team at Publicis Sapient is at the forefront of enterprise innovation, crafting scalable, end-to-end data and machine learning solutions across diverse global industries. Our multidisciplinary group blends expertise across cloud data engineering, data science pipelines, DevOps, and intelligent search systems. By leveraging unified compute platforms and advanced AI architectures like RAG and LLMs, we construct robust data foundations that turn complex information landscapes into actionable corporate intelligence.

About the Role

As a Senior Data Engineer specializing in Azure Big Data and AI Solutions (SAL2), you will:

• Lead AI & Data Engineering Innovations: Lead the design, implementation, and deployment of large-scale big data architectures, analytical frameworks, and self-service ETL platforms on the Azure ecosystem.
• Strategic Cloud Data Migration: Drive complex data migration strategies from legacy environments and on-premises clusters into modern cloud-native solutions, ensuring minimal disruption and maximum architectural integrity.
• Advance Generative AI & ML Systems: Incorporate machine learning models and foundational Generative AI components—including RAG frameworks and Large Language Models (LLMs)—into production-grade data pipelines.
• Optimize Production Pipelines: Architect, optimize, and maintain automated end-to-end data pipelines using Azure Data Factory (ADF) and Azure Databricks to process distributed datasets with low latency and high availability.
• Promote Data-Driven Decision-Making: Conduct deeply detailed source data analysis, profiling, and schema mapping across large-scale enterprise data warehouses to maximize data quality and business insight.
• Foster DevOps Excellence: Embed standard DevOps practices and CI/CD automated deployment paradigms into data processing solutions, ensuring reliable and maintainable code delivery.

Key Responsibilities

• Develop, optimize, and scale automated data ingestion pipelines using Azure Data Factory (ADF) and Azure Databricks in batch mode.
• Design and maintain custom self-service analytics platforms and modular ETL frameworks using Python and PySpark.
• Perform exhaustive source data analysis, data mapping, and granular profiling to enforce structure on unorganized or multi-source datasets.
• Build, integrate, and operationalize machine learning and GenAI applications utilizing Azure Machine Learning, RAG strategies, and LLM orchestration tools.
• Deploy, monitor, and configure helper cloud web services and infrastructure nodes including Azure WebApps, Key Vaults, Function Apps, and Logic Apps.
• Proactively tune the performance of complex SQL queries, Synapse Data Warehouses, and Spark computation engines to minimize compute costs and execution times.
• Collaborate with engineering and product stakeholders to rapidly adapt to new data technologies and resolve intricate platform bottlenecks.

Your Skills & Experience:

• BS/MS in Computer Science, Mathematics, Engineering, Statistics, or another quantitative/computational field. Advanced degrees preferred.
• Expert-level proficiency in programming paradigms with Python and distributed cluster computing via PySpark and Apache Spark.
• Deep, hands-on architectural experience with Azure cloud services, specifically Azure Data Factory (ADF), Azure Databricks, and Azure Synapse / SQL Server.
• Demonstrated experience designing and maintaining large-scale enterprise Data Warehouses, covering data integration and schema design mapping.
• Solid command of SQL scripting, window functions, query execution planning, and the optimization of complex dataset manipulations.
• Familiarity with DevOps implementations, including CI/CD automated release management pipelines mapped for data applications.
• Practical understanding of building and deploying machine learning models or integrating GenAI capabilities (RAG, LLM tooling).
• Experience configuring helper microservices including Azure Function Apps, Logic Apps, and credential isolation protocols through Azure Key Vault.

Set Yourself Apart With:

• Proven track record migrating multi-terabyte on-premises clusters (Hadoop/legacy appliances) into cloud infrastructure ecosystems.
• Advanced experience productionizing scalable vector indexing, specialized embeddings, or high-performance inference endpoints for RAG systems.
• Strong leadership footprint or mentorship history driving technical agility and modern coding patterns within data-centric teams.

PERSONAL ATTRIBUTES

• Strong written, verbal, and interpersonal communication skills.
• Proven problem-solving skills with a high degree of adaptability to learn new and emerging data technologies.
• Collaborative, self-starter mindset requiring minimal supervision while executing complex tasks under high agility.
• Ability to effectively balance, prioritize, and manage multiple projects without sacrificing delivery quality.

EXPERIENCE

5.5 to 8 years of total experience with a focus on cloud big data and analytical model engineering.

ABOUT US

Publicis Sapient is a digital transformation partner helping established organizations get to their future, digitally- enabled state, both in the way they work and the way they serve their customers. We help unlock value through a start-up mindset and modern methods, fusing strategy, consulting and customer experience with agile engineering and problem-solving creativity. As digital pioneers with 20,000 people and 53 offices around the globe, our experience spanning technology, data sciences, consulting and customer obsession - combined with our culture of curiosity and relentlessness - enables us to accelerate our clients' businesses through designing the products and services their customers truly value. Publicis Sapient is the digital business transformation hub of Publicis Groupe. For more information, visit publicissapient.com