Databricks Data Engineer

Virtusa; Bengaluru Urban; On-site; INR 1,200,000 - 2,400,000; Full time

Job summary

Virtusa is seeking a Databricks Data Engineer to design, develop, and optimize scalable data pipelines supporting enterprise analytics, AI/ML, and Agentic AI solutions.

The ideal candidate will have hands-on experience with Azure Databricks, Spark (PySpark), Spark SQL, and Delta Lake for building batch and streaming pipelines, along with strong knowledge of Azure Data Lake Storage, Unity Catalog, and Kafka/Event Hubs.

Qualifications

• Hands-on experience with Azure Databricks and Spark for batch and streaming pipelines.
• Strong knowledge of Delta Lake and Lakehouse patterns.
• Proficiency in Python and SQL for data development.
• Experience with data governance, Unity Catalog, and cloud data storage.

Responsibilities

• Design, develop, and optimize scalable data pipelines for analytics and AI/ML.
• Collaborate with data scientists and MLOps to deliver governed data solutions.
• Tune Spark workloads and ensure data quality across pipelines.

Skills

Azure Databricks

Apache Spark (PySpark)

Spark SQL

Delta Lake

Python

SQL

Git-based CI/CD

Terraform

Azure Data Lake Storage

Unity Catalog

Kafka/Event Hubs

ETL/ELT

Lakehouse architecture

Data quality

Azure cloud

Spark performance tuning

Tools

Terraform

Git

CI/CD pipelines

Job description

"We are seeking a Databricks Data Engineer to design, develop, and optimize scalable data pipelines supporting enterprise analytics, AI/ML, and Agentic AI solutions."

The ideal candidate should have strong hands-on experience with Azure Databricks, Apache Spark (PySpark), Spark SQL, and Delta Lake for building batch and streaming data pipelines.

Experience with Azure Data Lake Storage, Unity Catalog, Kafka/Event Hubs, and data integration patterns is essential. The role requires expertise in ETL/ELT development, Lakehouse architecture, and data quality best practices.

Candidates should be proficient in Python, SQL, Git-based CI/CD, and Terraform.

Experience with Azure cloud services and performance tuning of Spark workloads is highly desirable.

The engineer will collaborate with data scientists, MLOps engineers, and platform teams to deliver reliable, governed, and scalable data solutions.

Strong analytical, troubleshooting, and communication skills are essential in a collaborative enterprise environment."