Azure Data Engineer

Virtusa; Bengaluru Urban; On-site; INR 1,200,000 - 1,900,000; Full time

Job summary

Virtusa is seeking a data engineering professional to design and implement end-to-end data solutions on Microsoft Azure, including data pipelines with ADF and scalable processing with Databricks. The role involves building data lakes, warehousing, and governance best practices while mentoring junior team members.

The candidate will work with stakeholders to translate requirements into technical solutions and ensure high availability of data systems in a fast-paced consulting environment.

Qualifications

• Experience with Azure Data Factory and Azure data engineering stacks.

Responsibilities

• Design and implement end-to-end data engineering solutions on Microsoft Azure.

Skills

Azure Data Factory

Azure Databricks

ADLS Gen2

Azure Synapse Analytics

HDInsight

Python/Scala

SQL & Data Modeling

ETL/ELT patterns

Distributed Data Processing

Azure DevOps CI/CD

Azure Monitor

Tools

Delta Lake

Power BI

Job description

You will work closely with clients, architects, and cross-functional teams to deliver robust data solutions that support analytics, reporting, and advanced data processing.

Responsibilities

• Design and implement end-to-end data engineering solutions on Microsoft Azure.
• Develop and maintain data pipelines using Azure Data Factory (ADF).
• Build scalable data processing frameworks using Azure Databricks (PySpark/Scala).
• Architect and manage data lakes using Azure Data Lake Storage (ADLS Gen2).
• Develop data warehousing and analytics solutions using Azure Synapse Analytics.
• Work with HDInsight for big data workloads (Hive, Spark, Kafka, etc.) where applicable.
• Optimize data workflows for performance, scalability, and cost-efficiency.
• Implement data governance, security, and compliance best practices.
• Collaborate with stakeholders to gather requirements and translate them into technical solutions.
• Provide technical leadership and mentor junior team members.
• Troubleshoot production issues and ensure high availability of data systems.

Required Skills & Qualifications

Core Technical Skills

• Strong experience with Azure Data Factory (ADF)
• Azure Databricks (PySpark, Spark SQL)
• Azure Data Lake Storage (ADLS Gen2)
• Azure Synapse Analytics
• HDInsight (Spark/Hadoop ecosystem)
• Proficiency in Python and/or Scala
• Strong SQL and data modeling skills (star/snowflake schema)
• Experience with ETL/ELT design patterns
• Knowledge of distributed data processing frameworks

Azure & Data Expertise

• Hands-on experience with Azure-native data services
• Understanding of data lake, lakehouse, and data warehouse architectures
• Experience implementing CI/CD pipelines (Azure DevOps)
• Familiarity with data security, RBAC, and encryption in Azure
• Experience with monitoring tools (Azure Monitor, Log Analytics)

Nice to Have

• Experience with Delta Lake / Lakehouse architecture
• Exposure to real-time data processing (Kafka/Event Hubs)
• Knowledge of Power BI or other visualization tools

Certifications

• Microsoft Certified Azure Data Engineer Associate

Soft Skills

• Strong problem-solving and analytical thinking
• Excellent communication and stakeholder management skills
• Ability to work in a fast-paced consulting environment
• Leadership and mentoring capabilities

Roles & Expectations (T2 Senior Consultant Level)

• Independently handle medium to large-scale projects