Senior Data Engineer (Databricks)

Virtusa; Bengaluru; Vor Ort; INR 2.500.000 - 4.200.000; Vollzeit

Zusammenfassung

Virtusa in Bengaluru seeks an experienced Data Engineering Architect to drive end-to-end data architectures using Databricks and Snowflake. You will design, optimize, and deploy streaming and batch pipelines on AWS, mentor teams, and collaborate with AI/DS groups to enable analytics and ML workflows.

The role emphasizes architectural leadership, data governance, and scalable solutions in a cloud-first environment, with a focus on real-time data processing and API-driven orchestration.

Qualifikationen

• B Tech/ M Tech in a relevant field.
• 7-10 years of work experience in data engineering or analytics.
• Experience with data architectures and cloud data ecosystems.

Aufgaben

• End-to-end design of scalable, secure data architectures with Databricks and Snowflake.
• Architect and optimize streaming and batch data pipelines (ETL/ELT).
• Deploy scalable enterprise data platform components in AWS.
• Design robust data ingestion frameworks using Databricks APIs and REST/GraphQL.
• Mentor engineering teams on data best practices.

Kenntnisse

Spark

Databricks

Kafka

Stream Processing

SQL

Python

Snowflake

Ausbildung

B Tech/ M Tech

Jobbeschreibung

Job Requirements

Mandatory Skills

Spark,Databricks Engineer,Kafka,Stream Processing,SQL

Development,Python,Snowflake

Job Description

Architecture & Strategy: End-to-end design of scalable, secure, and highly available

data architectures leveraging modern cloud data ecosystems (Databricks and

Snowflake).

Pipeline Engineering: Architect, optimize, and oversee the deployment of reliable

streaming and batch data pipelines (ETL/ELT) to process complex, large-scale

datasets.

Cloud Architecture: Architect and deploy scalable enterprise data platform

components natively within the AWS ecosystem, ensuring tight integration with core

security, IAM, and networking protocols.

API Ingestion & Orchestration: Design and implement robust data ingestion

frameworks leveraging Databricks APIs and external REST/GraphQL APIs for

automated workflows, platform orchestration, and data delivery.

Real-time Processing: Design and implement robust frameworks for real-time data

ingestion and processing to solve business-critical, low-latency use cases.

Hybrid Data Modelling: Harmonize "old school" relational data warehousing patterns

(Kimball/Inmon, Star/Snowflake schemas) with unstructured/semi-structured modern

paradigms.

Technical Leadership: Act as a core problem-solver for complex data bottlenecks,

provide technical governance, and mentor engineering teams on data best practices.

AI Integration: Collaborate with Data Science and AI teams to architect data layers that

seamlessly support LLMs, Machine Learning pipelines, and advanced analytics

solutions.

Education Qualificaiton

B Tech/ M Tech

Roles & Responsibilities

Architecture & Strategy: End-to-end design of scalable, secure, and highly available data

architectures leveraging modern cloud data ecosystems (Databricks and Snowflake).

Pipeline Engineering: Architect, optimize, and oversee the deployment of reliable streaming

and batch data pipelines (ETL/ELT) to process complex, large-scale datasets.

Cloud Architecture: Architect and deploy scalable enterprise data platform components

natively within the AWS ecosystem, ensuring tight integration with core security, IAM, and

networking protocols.

API Ingestion & Orchestration: Design and implement robust data ingestion frameworks

leveragin

Work Experience

7-10Years