Azure Databricks Data Engineer (Power BI)

Tata Consultancy Services; Kolkata District, Pune District, Bengaluru; On-site; INR 1,500,000 - 3,200,000; Full time

Job summary

Tata Consultancy Services is seeking an experienced Azure Databricks Data Engineer to design and develop scalable cloud data pipelines and enterprise reporting within the Azure ecosystem.

The role emphasizes ETL/ELT development, data platform design, and BI reporting using Power BI, with hands-on work on large data marts and analytics platforms in a enterprise context.

Qualifications

• 6-8 years of experience in Data Engineering and Analytics.
• Hands-on experience with Azure Databricks.
• Power BI development experience.
• Expertise in Azure Data Factory (ADF).
• Strong Python and PySpark development skills.
• Experience with SQL and T-SQL programming.
• Good understanding of Informatica.
• Experience building ETL and data integration solutions.
• Experience with Azure Synapse Analytics or similar cloud data platforms.
• Strong knowledge of Data Warehousing and Data Mart concepts.

Responsibilities

• Design, develop, and maintain ETL/ELT processes using Azure Databricks, Azure Data Factory, and Azure Synapse.
• Build scalable data ingestion and transformation pipelines using Python and PySpark.
• Establish cloud connectivity across Azure platforms including ADLS, ADF, Synapse Analytics, and Databricks.
• Develop robust data integration solutions for structured and unstructured data sources.
• Optimize ETL processes for performance, scalability, and reliability.

Skills

Azure Databricks

Power BI

Azure Data Factory

Python

PySpark

SQL

T-SQL

Informatica

ETL/ELT

Azure Synapse

Data Warehousing

Data Marts

Tools

Azure Data Factory (ADF)

Power BI

Informatica

Job description

We are looking for an experienced Azure Databricks Data Engineer with strong expertise in Azure Databricks, Power BI, Azure Data Factory, SQL, Python, PySpark, and Informatica. The ideal candidate will be responsible for designing and developing scalable ETL solutions, modern cloud data pipelines, and enterprise reporting platforms within the Azure ecosystem.

The role requires hands-on experience with large-scale data marts, Azure analytics services, ETL frameworks, database optimization, and business intelligence reporting.

Key Responsibilities

Data Engineering & ETL Development

• Design, develop, and maintain ETL/ELT processes using Azure Databricks, Azure Data Factory, and Azure Synapse.
• Build scalable data ingestion and transformation pipelines using Python and PySpark.
• Establish cloud connectivity across Azure platforms including ADLS, ADF, Synapse Analytics, and Databricks.
• Develop robust data integration solutions for structured and unstructured data sources.
• Optimize ETL processes for performance, scalability, and reliability.

Azure Data Platform Development

• Build and maintain cloud-based data engineering solutions within Azure.
• Support data lake, data mart, and analytical platform initiatives.
• Design reusable frameworks for data processing and orchestration.
• Implement data quality and governance controls across platforms.

Business Intelligence & Reporting

• Develop and maintain interactive Power BI dashboards and reports.
• Deliver reporting and analytical solutions aligned with business requirements.
• Support KPI reporting, management dashboards, and enterprise analytics solutions.
• Collaborate with business teams to understand reporting and visualization requirements.

Database & Data Mart Management

• Design and optimize SQL queries, stored procedures, and database objects.
• Work with large-scale SQL data marts and enterprise databases.
• Perform database tuning, performance optimization, and troubleshooting.
• Support database backup, archival, security, access management, and capacity planning.

Operational Support

• Troubleshoot ETL, reporting, database, and platform-related issues.
• Support testing activities and production deployments.
• Collaborate with cross-functional teams to ensure successful project delivery.
• Maintain technical documentation and data platform standards.

Required Skills

• 6-8 years of experience in Data Engineering and Analytics.
• Strong hands-on experience with Azure Databricks.
• Strong Power BI development experience.
• Expertise in Azure Data Factory (ADF).
• Strong Python and PySpark development skills.
• Experience with SQL and T-SQL programming.
• Good understanding of Informatica.
• Experience building ETL and data integration solutions.
• Experience with Azure Synapse Analytics or similar cloud data platforms.
• Strong knowledge of Data Warehousing and Data Mart concepts.
• Experience working with large enterprise datasets and analytics platforms.

Preferred Skills

• Azure Synapse Analytics.
• Azure Data Lake Storage (ADLS Gen2).
• Data Modeling and Data Warehouse Design.
• Performance Tuning and Query Optimization.
• Azure Analytics Services.
• Banking and Financial Services domain experience.
• Cloud Data Migration and Modernization projects.

Desired Candidate Profile

• Strong analytical and problem-solving abilities.
• Excellent verbal and written communication skills.
• Ability to navigate large datasets and deliver effective solutions.
• Flexible, proactive, and self-motivated approach.
• Strong collaboration and stakeholder management skills.
• Experience working in Agile environments.