Microsoft Fabric Data Engineer

Acme Services Private Limited; Hyderabad; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

Acme Services Private Limited in Hyderabad, India is seeking a Data Engineer to design, build, and maintain scalable data pipelines using Microsoft Fabric, Azure Data Factory, and related services. The role involves implementing Medallion Architecture and optimizing data workflows across Databricks and Snowflake as needed.

The candidate will develop robust PySpark-based solutions, comply with data governance standards, and collaborate with Data Architects, BI Developers, and analysts to deliver

Qualifications

• Experience in designing scalable data pipelines using modern data platforms.
• Hands-on experience with Fabric Lakehouse, Data Factory, and related services.
• Strong knowledge of ETL/ELT, governance, and data quality practices.

Responsibilities

• Design, develop, and maintain scalable data pipelines and architectures.
• Implement Medallion Architecture (Bronze, Silver, Gold) for governed data processing.
• Collaborate with cross-functional teams to deliver end-to-end analytics solutions.

Skills

Microsoft Fabric

PySpark

Spark SQL

Data Factory

Databricks

Tools

SQL

Azure Data Factory

Notebooks Gen2

Job description

Job Description:

• Design, develop, and maintain scalable Microsoft Fabric data engineering solutions using Fabric Lakehouse, Data Warehouse, Data Factory, and related services.




• Build and manage modern data platforms leveraging Microsoft OneLake as a centralized data lake.




• Develop robust ETL/ELT pipelines for batch and near-real-time data ingestion from various enterprise and cloud sources.




• Implement Medallion Architecture (Bronze, Silver, Gold) for structured and governed data processing.




• Develop data transformation and processing solutions using PySpark, Spark SQL, SQL, and other relevant technologies.




• Design and implement data pipelines using Microsoft Fabric Data Factory and Azure Data Factory (ADF).




• Integrate data from multiple sources including relational databases, APIs, files, cloud applications, and enterprise systems.




• Work with Fabric Lakehouse, Warehouse, Dataflows Gen2, Notebooks, Pipelines, and semantic models to deliver end-to-end analytics solutions.




• Develop and optimize data solutions using Databricks and Snowflake where required.




• Implement data quality, validation, monitoring, and reconciliation processes across data pipelines.




• Apply data governance, security, access control, lineage, and compliance best practices.




• Work with Microsoft Purview and related governance capabilities to support data cataloging, classification, lineage, and discovery.




• Optimize data pipelines, Spark workloads, SQL queries, storage, and compute resources for improved performance and cost efficiency.




• Implement monitoring, logging, error handling, alerting, and recovery mechanisms for production data pipelines.




• Collaborate with Data Architects, BI Developers, Data Scientists, Business Analysts, and other stakeholders to understand requirements and deliver scalable data solutions.




• Participate in architecture discussions, technical design, code reviews, testing, deployment, and production support.




• Follow CI/CD and DevOps practices for data engineering development and deployment.




• Troubleshoot complex data pipeline, integration, performance, and production issues.