Senior Data Engineer (Pyspark, Microsoft Fabric.)

Quest Global; Chennai District; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Quest Global in Chennai is seeking a data engineer to design and implement scalable data processing solutions using PySpark within Microsoft Fabric. You will work on building robust ETL/ELT pipelines for large-scale datasets and collaborating with cross-functional teams to deliver reliable data products.

The role involves developing data ingestion, transformation, and aggregation workflows using Fabric Notebooks, Lakehouse, and Data Factory Pipelines to support analytics and decision-making

Qualifications

• Hands-on experience designing scalable data processing solutions with PySpark in Microsoft Fabric.
• Proven ability to build and optimize ETL/ELT pipelines for large-scale datasets using Spark.
• Experience developing data ingestion, transformation, and aggregation workflows using Fabric Notebooks, Lakehouse, and Data Factory Pipelines.

Skills

PySpark

Microsoft Fabric

Spark DataFrames

Spark SQL

ETL/ELT pipelines

Ingestion/Transformation

Tools

Fabric Notebooks

Lakehouse

Data Factory Pipelines

Job description

Job Requirements

At Quest Global, it’s not just what we do but how and why we do it that makes us different. With over 25 years as an engineering services provider, we believe in the power of doing things differently to make the impossible possible. Our people are driven by the desire to make the world a better place—to make a positive difference that contributes to a brighter future. We bring together technologies and industries, alongside the contributions of diverse individuals who are empowered by an intentional workplace culture, to solve problems better and faster.

We are known for our extraordinary people who make the impossible possible every day. Questians are driven by hunger, humility, and aspiration. We believe that our company culture is the key to our ability to make a true difference in every industry we reach. Our teams regularly invest time and dedicated effort into internal culture work, ensuring that all voices are heard.

We wholeheartedly believe in the diversity of thought that comes with fostering a culture rooted in respect, where everyone belongs, is valued, and feels inspired to share their ideas. We know embracing our unique differences makes us better, and that solving the worlds hardest engineering problems requires diverse ideas, perspectives, and backgrounds. We shine the brightest when we tap into the many dimensions that thrive across over 21,000 difference-makers in our workplace.

Work Experience

• Hands‑on experience in designing and developing scalable data processing solutions using PySpark in Microsoft Fabric.
• Experience building and optimizing ETL/ELT pipelines for processing large-scale structured and semi‑structured datasets using Spark DataFrames and Spark SQL.
• Experience developing data ingestion, transformation, and aggregation workflows using Fabric Notebooks, Lakehouse, and Data Factory Pipelines.