Senior Data Engineer (Azure Synapse)

Quantiphi Analytics Solutions Private Limited; Bengaluru; On-site; INR 1,500,000 - 2,100,000; Full time

Job summary

Quantiphi Analytics Solutions Private Limited is seeking a Senior Data Engineer (Azure Synapse) to design and deliver robust big data pipelines across cloud platforms. The role requires collaboration with cross-functional teams to implement scalable data ingestion and processing solutions in a healthcare-focused environment.

The ideal candidate brings 3+ years of hands-on data engineering, strong SQL and Python/PySpark skills, and deep Azure Synapse experience to enable mass-scale analytics and

Qualifications

• 3+ years of hands-on data engineering experience.
• Strong proficiency in SQL and Python/PySpark.
• Experience building big data solutions on cloud platforms.
• Production-grade systems experience and a track record of ongoing operations.
• ETL/ELT pipelines using Azure Data Factory and Synapse Pipelines.
• Deep Azure data services and distributed processing knowledge.
• Experience with Spark in Azure Synapse environments.
• Knowledge of ADLS Gen2 and data warehousing concepts.

Responsibilities

• Develop and maintain robust big data pipelines across Azure Synapse.
• Design cloud migration strategies for cloud and on-premise apps with Synapse integration.
• Troubleshoot distributed systems within Azure at scale.
• Build tools for ingestion and processing of terabytes to petabytes daily.
• Create next-gen storage and compute solutions for large customers.
• Collaborate with Infrastructure, Network, Engineering, DevOps and cloud customers.
• Develop tooling for automation, testing, monitoring and data ops.

Skills

Data engineering

SQL

Python/PySpark

Production systems

Big data pipelines

Tools

Azure Synapse Analytics

Azure Data Factory

Azure Data Lake Storage

Azure Synapse Spark

Azure Cosmos DB

Azure Data Explorer

Apache Airflow

Azure Event Hubs

Azure Service Bus

Kubernetes AKS

Kafka

Job description

While technology is the heart of our business, a global and diverse culture is the heart of our success. We love our people and we take pride in catering them to a culture built on transparency, diversity, integrity, learning and growth. If working in an environment that encourages you to innovate and excel, not just in professional but personal life, interests you- you would enjoy your career with Quantiphi!

Role: Senior Data Engineer (Azure Synapse)

Experience Level: 3 to 6 Years

Work location: Mumbai, Bangalore & Trivandrum

We are seeking a highly skilled and experienced Senior Data Engineer with a strong focus on Azure Synapse Analytics to join our innovative team within the healthcare domain. In this critical role, you will be instrumental in designing and delivering robust big data pipelines for both structured and unstructured data, operating across multiple geographies. Your expertise will directly contribute to helping healthcare organizations achieve their business goals through advanced data ingestion technologies, cloud services, and DevOps practices. You will collaborate closely with Architects from various specialties, including Cloud Engineering, Software Engineering, and ML Engineering, to create cutting‑edge platforms, solutions, and applications. These initiatives will address the latest trends in the healthcare industry, such as digital diagnosis, software as a medical product (SaMD), and AI marketplaces.

Role & Responsibilities

• Azure Synapse Expertise: Work with cloud engineers and customers to solve complex big data problems by developing utilities for migration, storage, and processing specifically leveraging Azure Synapse Analytics.
• Cloud Migration Strategy: Design and build a comprehensive cloud migration strategy for both cloud and on‑premise applications, with a strong emphasis on integrating with Azure Synapse.
• Troubleshooting & Optimization: Diagnose and troubleshoot complex distributed systems problems within the Azure ecosystem, developing solutions with a significant impact at massive scale, particularly within Synapse environments.
• Large-Scale Data Processing: Build tools to ingest and jobs to process several terabytes or petabytes per day, optimizing performance and cost within Azure Synapse.
• Next-Gen Storage & Compute: Design and develop next‑generation storage and compute solutions for several large customers, utilizing the full capabilities of Azure Synapse.
• Cross-Functional Collaboration: Communicate effectively with a wide set of teams, including Infrastructure, Network, Engineering, DevOps, SiteOps teams, and cloud customers, ensuring seamless integration and delivery.
• Advanced Tooling: Build advanced tooling for automation, testing, monitoring, administration, and data operations across multiple Azure Synapse workspaces and associated cloud clusters.

Skill Must Skills

• 3+ years of hands-on Data Engineering experience with strong knowledge of data structures, distributed systems, data modeling, and databases.
• Strong proficiency in SQL and hands‑on experience with Python/PySpark.
• Extensive experience in developing Big Data solutions (migration, storage, processing) with a proven track record of implementing solutions on cloud platforms.
• Experience building and supporting large‑scale systems in a production environment.
• Strong experience developing ETL/ELT pipelines using Azure Data Factory (ADF) and Synapse Pipelines.
• Cloud Platforms: Deep expertise in Azure, specifically Azure Synapse Analytics (SQL Pools, Spark Pools, Data Explorer, Pipelines).
• Big Data Distributions: Experience with Azure Synapse Analytics, Azure Data Lake Storage (ADLS Gen2).
• Distributed Processing Frameworks: Strong experience with Apache Spark (especially within Azure Synapse Spark Pools).
• Database/Warehouse: Azure Synapse SQL Pools, Azure Cosmos DB, Azure Data Explorer, and other relevant Azure data services.
• Orchestration Frameworks: Azure Data Factory, Apache Airflow, or similar.
• Message/Event Solutions: Azure Event Hubs, Azure Service Bus, Kafka.
• Container Orchestration (Good to have): Kubernetes (Azure Kubernetes Service - AKS).

If you like wild growth and working with happy, enthusiastic over‑achievers, you'll enjoy your career with us! Thank you for your interest in Quantiphi! We are a team that dreams together and collaborates to ensure success. We are currently looking for exceptional individuals who can join us and contribute to a fun, diverse and hybrid work culture. As a part of the Quantiphi family, you will get ample opportunities to learn, grow and interact with colleagues from varied experience and backgrounds around the globe. Quantiphi is an award‑winning AI‑first digital engineering company driven by the desire to reimagine and realize transformational opportunities at the heart of business. We solve the toughest and most complex business problems with the latest and cutting‑edge technologies and to make this happen, we have a vibrant, diverse and talented set of professionals we proudly refer to as the Quantiphi family. Our culture is built on transparency, diversity, integrity, learning and growth. We are committed to provide our teams with an environment that helps them to learn, grow and flourish in their professional as well as personal lives.