Azure Data Engineer

Ignitho; Chennai District; Hybrid; INR 900,000 - 1,300,000; Full time

Job summary

Ignitho is seeking a Senior Azure Data Engineer with 7+ years of data engineering experience and hands-on expertise in Microsoft Fabric, Azure Synapse, or Databricks. You will design, build and optimize scalable data pipelines and Lakehouse solutions on the Azure platform.

You'll work across Fabric components (OneLake, Data Factory, Lakehouse, Notebooks, Spark, Power BI Direct Lake) and collaborate with architects, analysts and business stakeholders, mentoring junior engineers and driving best

Qualifications

• 7+ years in data engineering with Fabric or Synapse/Databricks.
• Strong knowledge of Fabric Lakehouse, OneLake, and Data Factory pipelines.
• Proficient in PySpark, Spark SQL, Delta Lake.
• SQL and Python programming for distributed data processing.
• Experience with Power BI Direct Lake semantics.
• CI/CD and Git-based deployment on Fabric.
• Excellent collaboration and mentoring abilities.

Responsibilities

• Design, build, and maintain end-to-end data pipelines on Fabric.
• Develop Lakehouse and Warehouse architectures on OneLake.
• Implement Delta Lake, partitioning, and schema evolution.
• Migrate workloads from other platforms to Fabric where applicable.
• Ensure data quality, governance, and security controls.
• Mentor junior engineers and perform code reviews.

Job description

Senior Azure Data Engineer

Experience Required: 7- 10+ years overall, with 2+ years hands-on in Microsoft Fabric or equivalent experience in Azure Synapse/Azure Databricks with strong Fabric readiness.

Role Summary

We are seeking a Senior Data Engineer with strong Microsoft Fabric expertise to design, build, and optimize scalable data pipelines and Lakehouse solutions on Azure. This role involves hands‑on engineering across the Microsoft Fabric platform, including OneLake, Data Factory, Lakehouse, Warehouse, Notebooks, Spark, and Power BI Direct Lake, while collaborating with architects, analysts, and business stakeholders.

Key Responsibilities

• Design, build, and maintain end‑to‑end data pipelines in Microsoft Fabric using Data Factory pipelines/dataflows, Notebooks, and Spark jobs.
• Build and manage Lakehouse and Warehouse architectures on OneLake, implementing Bronze/Silver/Gold layers.
• Develop and optimize PySpark/Spark SQL transformations within Fabric Notebooks.
• Implement Delta Lake tables with proper partitioning, compaction, and schema evolution handling.
• Integrate real‑time data using Fabric Real‑Time Intelligence, Eventstream, and KQL Database.
• Build semantic models and enable Direct Lake mode in Power BI.
• Migrate existing workloads from Synapse Analytics / Azure Databricks / SSIS to Microsoft Fabric where applicable.
• Implement data quality, validation, and error‑handling frameworks.
• Apply data governance and security controls using Fabric workspace roles, OneLake security, and Microsoft Purview integration.
• Optimize Fabric capacity utilization and cost.
• Write clean, modular, reusable code and maintain version control using Git integration with Fabric workspaces.
• Collaborate with data architects on solution design and participate in code reviews.
• Mentor junior engineers.
• Troubleshoot pipeline failures, performance bottlenecks, and data discrepancies in production.
• Document data flows, transformation logic, and pipeline dependencies.

Required Skills & Experience

• 7+ years in data engineering with strong hands‑on experience in Microsoft Fabric, or equivalent depth in Synapse/Databricks with demonstrated Fabric upskilling.
• Strong working knowledge of:
• Fabric Lakehouse & Warehouse
• OneLake
• Fabric Data Factory pipelines and Dataflows Gen2
• Fabric Notebooks
• PySpark / Spark SQL
• Delta Lake
• Power BI, especially Direct Lake semantic modeling
• Strong SQL skills, including complex queries, stored procedures, and performance tuning.
• Strong Python/PySpark programming for distributed data processing.
• Experience with Fabric Real‑Time Intelligence, Eventstream, KQL DB, or equivalent technologies such as Event Hubs/Stream Analytics.
• Understanding of dimensional modeling Star/Snowflake schema and Medallion Architecture.
• Familiarity with Git‑based version control and CI/CD for Fabric deployment pipelines.
• Knowledge of Fabric security model workspace roles, item‑level permissions, and OneLake data access.
• Prior experience with Azure Data Factory, Azure Synapse Analytics, or Azure Databricks.

Preferred Qualifications

• Microsoft Certified: DP-600 (Fabric Analytics Engineer Associate) or DP-203 (Azure Data Engineer Associate).
• Experience migrating workloads from Synapse/Databricks to Microsoft Fabric.
• Exposure to Microsoft Purview for data governance and lineage.
• Familiarity with Azure DevOps and Fabric Git integration.
• Experience with Semantic Link (SemPy) or Fabric data science/ML capabilities.
• Life Sciences domain experience is preferred.

Soft Skills

• Strong problem‑solving skills and ability to independently troubleshoot complex pipeline issues.
• Good communication skills for collaboration with architects, analysts, and business teams.
• Detail‑oriented approach to data quality and reliability.
• Ability to adapt quickly to Microsoft's evolving Fabric feature set.
• Strong fundamentals in Spark, SQL, data modeling, and distributed data processing.