Data Engineering Specialist_104337

MyCareernet; Bengaluru; On-site; INR 5,000,000 - 7,000,000; Full time

Benefits offered by this job

Health insurance

Job summary

MyCareernet in Bengaluru, India, is seeking a senior data engineer to design and deliver end-to-end data solutions on Microsoft Azure and Fabric. You will build scalable data pipelines, implement lakehouse architectures with Bronze/Silver/Gold layers, and align data models with business needs.

The role requires 7+ years in data engineering on Microsoft platforms, strong SQL, PySpark, and DataOps practices including CI/CD, monitoring, and governance.

Qualifications

• 7+ years data engineering experience within the Microsoft platform.
• Strong experience with Azure Data Factory and data integration patterns using APIs, Python scripts, etc.
• Strong proficiency in SQL and data modelling for analytics-ready solutions.
• Experience with Azure cloud data platforms and modern data architectures.
• Knowledge of Microsoft Fabric including lakehouse concepts and Power BI integration.
• Experience with PySpark / Spark-based data processing (Databricks or Fabric).
• Understanding of DataOps concepts including CI/CD, automated testing, monitoring, and data lifecycle management.
• Experience designing medallion architectures (Bronze/Silver/Gold) and lakehouse patterns.

Responsibilities

• Design and deliver end-to-end data solutions across Microsoft Azure and Fabric platforms.
• Build and maintain scalable data pipelines using Azure Data Factory and Fabric.
• Engineer Fabric Lakehouse architectures with Bronze/Silver/Gold layers.
• Design data models and data products aligned to business needs.
• Develop PySpark/Spark transformations in Fabric Notebooks.
• Integrate data from on-premise, Azure, and SaaS sources.
• Ensure data quality, governance, reliability across all data products.
• Implement DataOps practices including CI/CD, monitoring, and version control.

Skills

Azure Data Factory

Python / PySpark

SQL

Databricks / Fabric

Data modeling

DataOps

Medallion architecture

Power BI integration

Education

B.Tech / B.E.

M.Tech

BCA

B.Sc.

MSc

MCA

Job description

Roles and Responsibilities:

• Design and deliver end to end data solution across Microsoft Azure and Fabric platforms.
• Build and maintain scalable, high performance data pipelines using Azure Data Factory and Fabric.
• Engineer Microsoft Fabric Lakehouse architectures using medallion layers (Bronze, Silver, Gold).
• Design and implement data models and data products aligned to business needs.
• Develop and optimise data transformations using PySpark/ Spark within Microsoft Fabric Notebooks.
• Integrate data from on-premise, Microsoft Azure and SaaS platforms.
• Ensure data quality, performance, governance and reliability across all data products.
• Implement data operations practices including CI/CD, monitoring and version control.

Skills Required:

• 7+ years data engineering experience within the Microsoft platform.
• Strong experience with Azure Data Factory and modern data integration patterns through native connections, APIs, Python scripts etc.
• Strong proficiency in SQL and data modelling for analytics-ready solutions.
• Experience with Azure cloud data platforms and modern data architectures.
• Knowledge or experience of Microsoft Fabric, including lakehouse / warehouse concepts and Power BI integration.
• Experience with PySpark /Spark-based data processing (e.g. Databricks or Fabric).
• Understanding of DataOps principles, including CI/CD, automated testing, monitoring, and data lifecycle management.
• Experience designing medallion (Bronze/Silver/Gold) architectures, lakehouse patterns, and batch/near-real-time processing.
• Microsoft Fabric and Azure certifications are a plus.
• Strategic agility
• Openness to Learning
• Business acumen
• Communications skills
• Ethics & Values
• Compassion
• Self-Development
• Priority Setting
• Drive for Results

Education:

B.Tech, B.Tech-M.Tech Dual Degree, M.Tech, BCA, B.Sc., M.Sc., or MCA in any branch.