Senior Data Engineer- MS Fabric

Nihilent; Kolkata District, Pune District, Chennai District; On-site; INR 1,500,000 - 2,500,000; Full time

Job summary

Nihilent in Kolkata is seeking an experienced Azure Data Engineer to design and implement enterprise data warehouse solutions, leveraging Microsoft Fabric and PySpark.

You will optimize ETL/ELT processes, model data with star and snowflake schemas, and collaborate in Agile teams to deliver reliable data pipelines and insightful dashboards.

Qualifications

• Experience in data warehousing architecture and best practices.
• Hands-on EDW design and implementation experience.
• Strong dimensional modeling using Star/Snowflake schemas.
• ETL/ELT design and development expertise.
• Proficient SQL programming and query tuning.
• Experience with Delta Lake and Medallion architecture.
• Knowledge of metadata-driven ETL/ELT frameworks.

Responsibilities

• Design and implement end-to-end data warehouse solutions.
• Collaborate on solution design, architecture reviews, and technical discussions.
• Optimize ETL/ELT pipelines for performance and reliability.
• Develop data transformation logic using PySpark and SQL.
• Triage production data issues and drive root-cause analysis.
• Participate in CI/CD and DevOps practices for data pipelines.

Skills

Data warehousing concepts

EDW design

Dimensional modeling

ETL/ELT design

SQL

Data quality

Problem solving

Agile/Scrum

Tools

Microsoft Fabric

Lakehouse

Data Factory

Data Pipelines

Notebooks (PySpark)

Warehouse

Semantic Models

Power BI

Job description

Experience

• Overall 6+ years of experience in Azure Data Engineering and Data Warehousing.
• At least 2-3 years of hands-on experience with Microsoft Fabric.

Technical Skills

• Strong understanding of Data Warehousing concepts, architecture, and best practices.
• Hands-on experience in designing and implementing Enterprise Data Warehouse (EDW) solutions.
• Good experience in Dimensional Data Modeling using Star and Snowflake schema techniques.
• Strong knowledge of ETL/ELT design and implementation.
• Hands-on experience with Microsoft Fabric components, including:
• Lakehouse
• Data Factory
• Data Pipelines
• Notebooks (PySpark)
• Warehouse
• Semantic Models
• Power BI Reporting (optional)
• Good experience in developing data transformation logic using PySpark and SQL.
• Strong SQL programming and query optimization skills.
• Experience working with Delta Lake tables and Medallion Architecture (Bronze, Silver, Gold).
• Experience with Metadata-Driven ETL/ELT Frameworks.
• Understanding of Incremental Data Loading, CDC (Change Data Capture), and SCD (Slowly Changing Dimensions).
• Experience in data validation, reconciliation, and end-to-end testing.
• Strong understanding of data quality frameworks and validation techniques.
• Experience in performance tuning and optimization of ETL pipelines and SQL queries.
• Ability to troubleshoot and resolve production data issues within SLA timelines.
• Experience in monitoring data pipelines, job failures, and root cause analysis.

Azure & DevOps (Optional )

• Hands-on experience with Azure DevOps for source code management and CI/CD implementation.
• Experience creating and maintaining build and release pipelines.
• Knowledge of Git branching strategies and version control best practices.
• Experience in deployment automation across Development, UAT, and Production environments.

Data Integration

• Experience integrating data from multiple source systems such as SQL Server, Oracle, SAP, REST APIs, and flat files.
• Experience with incremental and full-load data ingestion patterns.
• Knowledge of handling structured and semi-structured data (JSON, XML, CSV, Parquet).

Project Experience

• Hands-on experience in at least one end-to-end Data Warehouse implementation project, from requirements gathering through production deployment and hypercare support.
• Experience participating in solution design, technical discussions, and architecture reviews.
• Ability to estimate development effort and prepare technical design documents.

Soft Skills

• Strong analytical and problem-solving skills.
• Excellent debugging and troubleshooting abilities.
• Good communication and stakeholder management skills.
• Ability to work independently and collaboratively within Agile/Scrum teams.
• Willingness to mentor junior team members and participate in knowledge-sharing sessions.

Preferred Skills

• Microsoft Fabric Certification (DP-700 or equivalent) is preferred.
• Knowledge of Power BI and Semantic Models.
• Exposure to Azure Data Factory, Azure Synapse Analytics, or Databricks/PySpark is mandatory .
• Experience working in Retail, Manufacturing, or Financial Services domains is desirable