Sr. Databricks Architect

Weekday AI (YC W21); Pune District; Hybrid; INR 2,500,000 - 3,500,000; Full time

Job summary

A leading technology consulting firm is seeking an experienced Data Architect to join their team. The ideal candidate will have over 14 years of IT experience, particularly in data warehousing and ETL projects, with strong expertise in Azure Databricks and PySpark. The role involves designing data solutions, managing client relationships, and leading technical discussions. This position requires strong analytical skills and the ability to mentor team members, ensuring impactful data-driven results.

Qualifications

• 14+ years of IT experience with 5+ years in data warehouse/ETL projects.
• Hands-on experience with Azure Databricks, PySpark, SQL, and Databricks Delta Lake Architecture.
• Experience in DevOps environments with Terraform, CircleCI, Git, and Agile methodology.

Responsibilities

• Design and develop modern data warehouse solutions using Databricks and the Azure stack.
• Provide innovative solutions in data engineering and analytics.
• Collaborate with DW/BI leads to understand ETL pipeline development requirements.

Skills

Azure Databricks

Medallion Architecture

Unity Catalog

PySpark

ETL

SQL

Data Warehousing

Client Management

Team Management

Education

Bachelor's or Master's degree in Computer Science or related field

Tools

Azure

Airflow

Apache Kafka

Hadoop

Terraform

CircleCI

Job description

This role is for one of the Weekday's clients.

Minimum Experience: 14 years.

Location: Bengaluru, Pune, NCR, Hyderabad, Chennai, Kolkata.

Job type: Full-time.

We are looking for an experienced Data Architect with strong expertise in Azure Databricks and modern data engineering practices. The ideal candidate combines technical mastery with a creative and analytical mindset to design forward-thinking data solutions, solve complex business problems, and maximize the value of data. Strong client management and team leadership skills are essential to drive collaboration and deliver impactful results.

Key Responsibilities

• Design and develop modern data warehouse solutions using Databricks and the Azure stack.
• Provide innovative solutions in data engineering and analytics to solve business problems.
• Collaborate with DW/BI leads to understand ETL pipeline development requirements.
• Identify gaps in existing pipelines, triage issues, and implement fixes.
• Work with business stakeholders to design data models that fulfill reporting needs.
• Mentor and guide team members to resolve technical challenges.
• Drive technical discussions with client architects and project teams.
• Orchestrate data pipelines using Airflow and ensure smooth end-to-end operations.

Skills & Qualifications

• Bachelor's or Master's degree in Computer Science or related field.
• 14+ years of IT experience with 5+ years in data warehouse/ETL projects.
• Hands‑on experience with Azure Databricks, PySpark, SQL, and Databricks Delta Lake Architecture.
• Strong understanding of Medallion Architecture and Unity Catalog.
• Experience with batch and streaming ETL (Kinesis) and event‑driven data pipelines.
• Familiarity with Apache Kafka, Hadoop ecosystem (Hive, Pig, Impala), and NoSQL databases (MongoDB, Cassandra, Neo4J).
• Skilled in handling structured and unstructured data, including imaging and geospatial datasets.
• Experience in DevOps environments with Terraform, CircleCI, Git, and Agile methodology.
• Proficient in RDBMS, complex SQL, PL/SQL, Unix shell scripting, performance tuning, and troubleshooting.
• Databricks Certified Data Engineer Associate/Professional (preferred).

Core Skills: Azure Databricks, Medallion Architecture, Unity Catalog, PySpark, ETL, SQL, Data Warehousing, Team & Client Management

Seniority level

Director

Employment type

Full-time

Job function

Other

Industries

IT Services and IT Consulting