Senior Data Engineer

Publicis Sapient; Bengaluru; On-site; INR 3,500,000 - 5,500,000; Full time

Job summary

Publicis Sapient's Data & AI team in Bengaluru seeks a Senior Data Engineer (SAL2) to drive Azure-based big data and GenAI solutions. You will design and deploy large-scale data pipelines, migrate data to cloud-native frameworks, and productionize ML/GenAI components with a focus on reliability and cost efficiency.

Join a global environment delivering data-driven insights through DevOps-driven data platforms, modern analytics, and scalable architectures across diverse industries.

Qualifications

• BS/MS in Computer Science, Mathematics, Engineering, Statistics, or another quantitative/computational field. Advanced degrees preferred.
• Expert-level proficiency in programming paradigms with Python and distributed cluster computing via PySpark and Apache Spark.
• Deep, hands-on architectural experience with Azure cloud services, specifically Azure Data Factory (ADF), Azure Databricks, and Azure Synapse / SQL Server.
• Demonstrated experience designing and maintaining large-scale enterprise Data Warehouses, covering data integration and schema design mapping.
• Solid command of SQL scripting, window functions, query execution planning, and the optimization of complex dataset manipulations.
• Familiarity with DevOps implementations, including CI/CD automated release management pipelines mapped for data applications.
• Practical understanding of building and deploying machine learning models or integrating GenAI capabilities (RAG, LLM tooling).
• Experience configuring helper microservices including Azure Function Apps, Logic Apps, and credential isolation protocols through Azure Key Vault.

Responsibilities

• Develop, optimize, and scale automated data ingestion pipelines using Azure Data Factory (ADF) and Azure Databricks in batch mode.
• Design and maintain custom self-service analytics platforms and modular ETL frameworks using Python and PySpark.
• Perform exhaustive source data analysis, data mapping, and granular profiling to enforce structure on unorganized or multi-source datasets.
• Build, integrate, and operationalize machine learning and GenAI applications utilizing Azure Machine Learning, RAG strategies, and LLM orchestration tools.
• Deploy, monitor, and configure helper cloud web services and infrastructure nodes including Azure WebApps, Key Vaults, Function Apps, and Logic Apps.
• Proactively tune the performance of complex SQL queries, Synapse Data Warehouses, and Spark computation engines to minimize compute costs and execution times.
• Collaborate with engineering and product stakeholders to rapidly adapt to new data technologies and resolve intricate platform bottlenecks.

Skills

Python

PySpark

Apache Spark

Azure Data Factory

Azure Databricks

Azure Synapse

SQL

CI/CD

GenAI tooling

RAG

Data Warehouses

Education

BS/MS in Computer Science

Tools

Azure Data Factory

Azure Databricks

Azure Synapse

SQL Server

Job description

At Publicis Sapient, we don't just build technology; we create transformative solutions. By integrating Generative AI, advanced machine learning, and modern cloud data architectures into our ecosystem, we're shaping a new era of intelligent, data-driven digital experiences for global clients. From engineering large-scale automated data pipelines to implementing self-service analytics frameworks and productionizing LLM architectures, we leverage cutting-edge technologies to solve complex business challenges.

Join us in driving innovation and delivering true business value to our customers through the strategic application of advanced big data systems, modern DevOps practices, and next-generation AI.

About the Team

The Data and AI Team at Publicis Sapient is at the forefront of enterprise innovation, crafting scalable, end-to-end data and machine learning solutions across diverse global industries. Our multidisciplinary group blends expertise across cloud data engineering, data science pipelines, DevOps, and intelligent search systems. By leveraging unified compute platforms and advanced AI architectures like RAG and LLMs, we construct robust data foundations that turn complex information landscapes into actionable corporate intelligence.

About the Role

As a Senior Data Engineer specializing in Azure Big Data and AI Solutions (SAL2), you will:

• Lead AI & Data Engineering Innovations: Lead the design, implementation, and deployment of large-scale big data architectures, analytical frameworks, and self-service ETL platforms on the Azure ecosystem.
• Strategic Cloud Data Migration: Drive complex data migration strategies from legacy environments and on-premises clusters into modern cloud-native solutions, ensuring minimal disruption and maximum architectural integrity.
• Advance Generative AI & ML Systems: Incorporate machine learning models and foundational Generative AI components—including RAG frameworks and Large Language Models (LLMs)—into production-grade data pipelines.
• Optimize Production Pipelines: Architect, optimize, and maintain automated end-to-end data pipelines using Azure Data Factory (ADF) and Azure Databricks to process distributed datasets with low latency and high availability.
• Promote Data-Driven Decision-Making: Conduct deeply detailed source data analysis, profiling, and schema mapping across large-scale enterprise data warehouses to maximize data quality and business insight.
• Foster DevOps Excellence: Embed standard DevOps practices and CI/CD automated deployment paradigms into data processing solutions, ensuring reliable and maintainable code delivery.

Key Responsibilities

• Develop, optimize, and scale automated data ingestion pipelines using Azure Data Factory (ADF) and Azure Databricks in batch mode.
• Design and maintain custom self-service analytics platforms and modular ETL frameworks using Python and PySpark.
• Perform exhaustive source data analysis, data mapping, and granular profiling to enforce structure on unorganized or multi-source datasets.
• Build, integrate, and operationalize machine learning and GenAI applications utilizing Azure Machine Learning, RAG strategies, and LLM orchestration tools.
• Deploy, monitor, and configure helper cloud web services and infrastructure nodes including Azure WebApps, Key Vaults, Function Apps, and Logic Apps.
• Proactively tune the performance of complex SQL queries, Synapse Data Warehouses, and Spark computation engines to minimize compute costs and execution times.
• Collaborate with engineering and product stakeholders to rapidly adapt to new data technologies and resolve intricate platform bottlenecks.

Your Skills & Experience:

• BS/MS in Computer Science, Mathematics, Engineering, Statistics, or another quantitative/computational field. Advanced degrees preferred.
• Expert-level proficiency in programming paradigms with Python and distributed cluster computing via PySpark and Apache Spark.
• Deep, hands-on architectural experience with Azure cloud services, specifically Azure Data Factory (ADF), Azure Databricks, and Azure Synapse / SQL Server.
• Demonstrated experience designing and maintaining large-scale enterprise Data Warehouses, covering data integration and schema design mapping.
• Solid command of SQL scripting, window functions, query execution planning, and the optimization of complex dataset manipulations.
• Familiarity with DevOps implementations, including CI/CD automated release management pipelines mapped for data applications.
• Practical understanding of building and deploying machine learning models or integrating GenAI capabilities (RAG, LLM tooling).
• Experience configuring helper microservices including Azure Function Apps, Logic Apps, and credential isolation protocols through Azure Key Vault.

Set Yourself Apart With:

• Proven track record migrating multi-terabyte on-premises clusters (Hadoop/legacy appliances) into cloud infrastructure ecosystems.
• Advanced experience productionizing scalable vector indexing, specialized embeddings, or high-performance inference endpoints for RAG systems.
• Strong leadership footprint or mentorship history driving technical agility and modern coding patterns within data-centric teams.

PERSONAL ATTRIBUTES

• Strong written, verbal, and interpersonal communication skills.
• Proven problem-solving skills with a high degree of adaptability to learn new and emerging data technologies.
• Collaborative, self-starter mindset requiring minimal supervision while executing complex tasks under high agility.
• Ability to effectively balance, prioritize, and manage multiple projects without sacrificing delivery quality.

EXPERIENCE

5.5 to 8 years of total experience with a focus on cloud big data and analytical model engineering.

ABOUT US

Publicis Sapient is a digital transformation partner helping established organizations get to their future, digitally- enabled state, both in the way they work and the way they serve their customers. We help unlock value through a start-up mindset and modern methods, fusing strategy, consulting and customer experience with agile engineering and problem-solving creativity. As digital pioneers with 20,000 people and 53 offices around the globe, our experience spanning technology, data sciences, consulting and customer obsession - combined with our culture of curiosity and relentlessness - enables us to accelerate our clients' businesses through designing the products and services their customers truly value. Publicis Sapient is the digital business transformation hub of Publicis Groupe. For more information, visit publicissapient.com