Data Engineer : Microsoft Fabric & Azure Databricks

ORION SYSTEMS; Pune District, Mumbai, Chennai District; Hybrid; INR 1,200,000 - 2,400,000; Full time

Job summary

ORION SYSTEMS in Pune seeks a data engineer to design, develop, and maintain scalable data pipelines using Microsoft Fabric. You will build Lakehouse architectures and manage data workflows with PySpark, SQL, and notebooks, delivering trusted datasets for analytics.

You will collaborate with analytics and BI teams, implement CI/CD, and ensure data quality, governance, and cost efficiency while optimizing performance and reliability.

Qualifications

• Experience designing and implementing scalable data pipelines.
• Hands-on experience with Lakehouse architectures and data warehousing concepts.
• Proficiency with PySpark, SQL, notebooks, and data transformation tools.

Responsibilities

• Design, develop, and maintain scalable data pipelines using Microsoft Fabric.
• Build and manage Lakehouse, Data Warehouse, OneLake, Dataflows, and Fabric Pipelines.
• Develop data transformation and processing solutions using PySpark, Notebooks, and SQL.
• Implement ETL/ELT workflows for structured and unstructured data ingestion.
• Optimize data models, performance, reliability, and cost efficiency.
• Ensure data quality, governance, security, and compliance standards.
• Collaborate with analytics, BI, and business teams to deliver trusted datasets.
• Implement CI/CD and automate deployment of data engineering solutions.

Skills

Problem solving

Stakeholder management

Communication skills

Tools

Microsoft Fabric

PySpark

SQL

Azure Data Factory

Azure Databricks

Delta Lake

Power BI

ADLS

OneLake

Job description

Role & responsibilities

• Design, develop, and maintain scalable data pipelines using Microsoft Fabric.
• Build and manage Lakehouse, Data Warehouse, OneLake, Dataflows, and Fabric Pipelines.
• Develop data transformation and processing solutions using PySpark, Notebooks, and SQL.
• Implement ETL/ELT workflows for structured and unstructured data ingestion.
• Optimize data models, performance, reliability, and cost efficiency.
• Ensure data quality, governance, security, and compliance standards.
• Collaborate with analytics, BI, and business teams to deliver trusted datasets.
• Implement CI/CD and automate deployment of data engineering solutions.

Preferred candidate profile

• Strong hands-on experience with Microsoft Fabric (Lakehouse, OneLake, Pipelines, Data Warehouse, Notebooks, Semantic Models).
• Expertise in PySpark and advanced SQL.
• Experience with Azure Data Factory, Azure Databricks, ADLS, and Delta Lake.
• Strong understanding of data modeling, ETL/ELT, and Lakehouse architecture.
• Experience with Power BI and analytics data platforms.
• Excellent problem-solving, stakeholder management, and communication skills