Senior Data Engineer

QUESTK2 Technologies; Hyderabad; Hybrid; INR 1,400,000 - 2,300,000; Full time

Job summary

QuestK2 Technologies is seeking a Senior Data Engineer to design, build, and optimize modern data platforms. You will own end-to-end pipelines from ingestion to analytics-ready models, ensuring data is reliable, secure, and scalable for product and business use.

You will work with the Data Platform/Analytics Engineering team to enable analytics, real-time ingestion, and cost-efficient data solutions using Azure Data Factory, Synapse, Databricks, and Fabric.

Qualifications

• 5+ years of experience in data engineering with strong SQL (T-SQL) and preferably Python/PySpark.
• Hands-on expertise in Azure Data Factory, Synapse, Databricks, and ADLS Gen2.
• Experience with Microsoft Fabric or readiness to work on it.

Responsibilities

• Design and build batch and streaming data pipelines using Azure Data Factory, Microsoft Fabric, and Azure Synapse Analytics.
• Develop large-scale data transformations using Azure Databricks / Spark (PySpark, Spark SQL) and Delta Lake.
• Build dimensional models and optimize Azure SQL, SQL Server, and Synapse for performance and cost.
• Enable analytics and self-service BI with Power BI (semantic models, DAX, dataflows, RLS).
• Implement real-time ingestion using Event Hubs, Stream Analytics, or Fabric Real-Time Intelligence.
• Set up CI/CD and Infrastructure as Code using Azure DevOps or GitHub Actions and Bicep/Terraform.
• Monitor pipelines, troubleshoot failures, and optimize cost and performance; mentor junior engineers.

Skills

Data engineering

SQL (T-SQL)

Python / PySpark

Azure / Databricks

Power BI basics

CI/CD awareness

Education

Bachelor's in Computer Science or Engineering

Tools

Azure Data Factory

Azure Synapse

Databricks

ADLS Gen2

Delta Lake

Git / DevOps

Terraform / Bicep

Job description

About the Role

We are looking for a Senior Data Engineer to design, build, and optimize modern data platforms. You will own end-to-end data pipelines, from ingestion to analytics-ready models, ensuring data is reliable, secure, and scalable for product and business use.

Team: Data Platform / Analytics Engineering

Key Responsibilities

• Design and build batch and streaming data pipelines using Azure Data Factory, Microsoft Fabric (Data Factory, Lakehouse, Warehouse), and Azure Synapse Analytics.
• Develop large-scale data transformations using Azure Databricks / Spark (PySpark, Spark SQL) and Delta Lake.
• Build dimensional models and optimize Azure SQL, SQL Server, and Synapse for performance and cost.
• Enable analytics and self-service BI with Power BI (semantic models, DAX, dataflows, RLS).
• Implement real-time ingestion using Event Hubs, Stream Analytics, or Fabric Real-Time Intelligence.
• Set up CI/CD and Infrastructure as Code using Azure DevOps or GitHub Actions and Bicep/Terraform.
• Monitor pipelines, troubleshoot failures, and optimize cost and performance; mentor junior engineers.

Required Skills

• 5+ years of experience in data engineering with strong SQL (T-SQL) and Preferably Python/PySpark.
• Hands-on expertise in Azure Data Factory, Synapse, Databricks, and ADLS Gen2.
• Experience with Microsoft Fabric or strong readiness to work on it.
• Strong data modeling skills: star/snowflake schemas, SCDs, ETL/ELT design patterns.
• Working knowledge of Power BI and the Azure security model.
• Experience with Git, CI/CD, and agile delivery.

Good to Have

• Azure certifications: DP-203, DP-600, DP-700, AZ-900, or AZ-104.
• Experience with Delta Lake optimization, Unity Catalog, and Kafka.
• Exposure to Microsoft Purview, Azure Monitor, and Log Analytics.
• Knowledge of data governance and compliance (GDPR, DPDP, SOC 2, ISO 27001).
• Experience in SaaS or product-based companies is a plus.