Sr Consultant Data Engineer

IRT Digital Analytics Solutions; Dadri, Greater Noida, Mumbai; On-site; INR 1,000,000 - 1,500,000; Full time

Job summary

IRT Digital Analytics Solutions in India is seeking a data engineer to design and optimize data pipelines using Microsoft Fabric and Azure services. You will build scalable data models, implement governance, and create interactive dashboards with Power BI for analytics and AI use cases.

The role focuses on real-time processing, RBAC/RLS security, and close collaboration with data engineers, analysts, and business stakeholders to define data strategies and ensure high-performance analytics

Responsibilities

• Design and implement data pipelines using Microsoft Fabric to extract, transform, and load (ETL) data from various sources.
• Develop and manage scalable data models in Microsoft Fabrics OneLake and integrate with Power BI datasets.
• Build interactive dashboards and reports using Power BI within Microsoft Fabric.
• Implement data governance policies, RBAC using Functions and RLS, manage access controls within Microsoft Fabric to ensure compliance and security.

Job description

Role & responsibilities

• Design and implement data pipelines using Microsoft Fabric to extract, transform, and load (ETL) data from various sources.
• Develop and manage scalable data models in Microsoft Fabrics OneLake and integrate with Power BI datasets.
• Build interactive dashboards and reports using Power BI within Microsoft Fabric.
• Implement data governance policies, RBAC using Functions and RLS, manage access controls within Microsoft Fabric to ensure compliance and security.
• Leverage Microsoft Fabric's Lakehouse for ingesting structured and unstructured data into data lake for analytics and AI use cases.
• Optimize data pipelines in ADF and queries, and reports to ensure high performance across analytics workloads.
• Collaborate with data engineers, analysts, and business stakeholders to define and implement data strategies using Microsoft Fabric.
• Integrate and create ETL pipelines in Microsoft Fabric with Azure services like Azure Data Factory, Azure Synapse, and Azure Machine Learning.
• Implement real-time data processing and streaming analytics using Microsoft Fabric's built-in capabilities.
• Run pyspark ETL queries on notebooks and perform data transformation using pyspark script.