Data Engineer Lead

Skillventory; Navi Mumbai, Mumbai; On-site; INR 900,000 - 1,500,000; Full time

Job summary

Skillventory in Navi Mumbai is seeking an experienced Data Engineer to design, develop and maintain end-to-end data pipelines using Azure Databricks and Azure Data services, enabling scalable analytics across the business.

You will build ETL/ELT workflows with Python, PySpark and SQL, integrate data from multiple sources, and optimize performance while ensuring data quality. You will collaborate with cross-functional teams and deliver insightful dashboards with Power BI.

Qualifications

• Experience with Azure Databricks and distributed data processing.
• Strong Python for data engineering tasks.
• Hands-on with Azure Data Services (ADF, ADLS, Synapse).
• Strong SQL and database concepts.
• Power BI development and dashboard creation.
• Data warehousing concepts (Star schema, Snowflake).
• Experience with large-scale data processing and performance tuning.
• Familiarity with Git and CI/CD pipelines.
• Strong analytical and problem-solving skills.

Responsibilities

• Design, develop, and maintain end-to-end data pipelines using Azure Databricks and Azure Data services.
• Build scalable ETL/ELT workflows with Python, PySpark, and SQL.
• Integrate data from multiple sources into data platforms.
• Optimize data processing performance and ensure data quality and reliability.
• Develop data models to support analytics and reporting.
• Create and manage dashboards and reports using Power BI.
• Collaborate with business stakeholders to deliver insights.
• Work with cross-functional teams (Data Ops, Architects, DevOps).
• Focus on end-to-end strategy, AI/ML integration, and regulatory reporting.
• Business & Compliance: Regulatory Reporting, BI, AI, ML Models.

Skills

Azure Databricks

Python

Azure Data Services

SQL

Power BI

Data warehousing

Git

CI/CD

Big data processing

Tools

Azure Databricks UI

Power BI Desktop

Job description

Role & responsibilities :

• Design, develop, and maintain end-to-end data pipelines using Azure Databricks and Azure Data services
• Build scalable and efficient ETL/ELT workflows using Python, PySpark, and SQL
• Integrate data from multiple sources (structured and unstructured) into data platforms
• Optimize data processing performance and ensure data quality, reliability, and consistency
• Develop and maintain data models to support analytics and reporting requirements
• Create and manage interactive dashboards and reports using Power BI, Azure Databricks
• Collaborate with business stakeholders to understand data requirements and deliver actionable insights
• Work closely with cross-functional teams (Data Ops, Architects, DevOps)
• Focuses on end-to-end strategy, AI/ML integration, and regulatory reporting capabilities.
• Business & Compliance: Regulatory Reporting,, BI, AI, ML Models.

Preferred candidate profile

• Strong experience with Azure Databricks and distributed data processing
• Proficiency in Python for data engineering tasks
• Hands-on experience with Azure Data Services (ADF, ADLS, Synapse, etc.)
• Expertise in SQL and database concepts.
• Solid experience in Power BI development, data modeling, and dashboard creation ,Databricks dashboard creation.
• Good understanding of data warehousing concepts (Star schema, Snowflake schema)
• Experience in handling large-scale data processing and performance tuning
• Familiarity with version control tools (e.g., Git) and CI/CD pipelines
• Strong analytical and problem-solving skills