Databricks Architect

Teamware Solutions Inc.; Bengaluru; Hybrid; INR 6,000,000 - 9,000,000; Full time

Job summary

Teamware Solutions invites an experienced Databricks Architect to join our analytics engineering practice in a hybrid work setup. You will design and implement scalable data lakehouse solutions using Azure Databricks, Delta Lake, and PySpark, while guiding a hands-on data engineering team.

You will own complex data models, metadata-driven Python frameworks, and end-to-end pipelines, collaborating with stakeholders from Life Sciences domains to deliver production-ready data solutions.

Qualifications

• 10+ years of IT experience with significant data engineering/architecture experience.
• Strong hands-on Azure Databricks, Python, PySpark, and SQL experience.
• Proven ability to architect and develop reusable, metadata-driven Python frameworks.
• Expert in enterprise data modeling and complex data pipelines.
• Experience leading Data Engineering teams while contributing to hands-on development.
• Working knowledge of Pharma/Life Sciences data and processes.
• Strong problem-solving, ownership, communication, and stakeholder management.
• Azure Databricks Lakehouse Architecture: Delta Lake, Unity Catalog, Workflows/Jobs, DAB, scalable secure production solutions.
• Advanced Python, PySpark, SQL with ability to develop, review, and optimize complex data solutions.
• Reusable Python framework development for ingestion, transformation, validation, orchestration, and automation.
• Data modeling across conceptual/logical/physical schemas; CDC, SCD Type 2, historical processing.
• Bronze/Silver/Gold lakehouse data architectures, Delta tables, scalable ETL/ELT pipelines.
• Real-time and batch processing using Spark Structured Streaming, Auto Loader, Event Hubs, CDF

Skills

10+ years IT experience

Azure Databricks

Python

PySpark

SQL

Reusable Python frameworks

Data modeling

Data engineering leadership

Pharma/Life Sciences data

Problem solving & Stakeholder mgmt

Databricks architecture

Advanced PySpark

Metadata-driven design

Delta Lake

Spark Structured Streaming

Job description

Greetings from Teamware Solutions a division of Quantum Leap Consulting Pvt. Ltd

Teamware Solutions (a division of Quantum Leap Consulting Pvt. Ltd.) is a 20+ year industry leader providing IT staffing and consulting services, supporting 4000+ professionals across India, the USA, the Middle East, and APAC for leading GCC clients.

About the Role

We are hiring a Databricks Architect

• Work Model: Hybrid
• Experience: 10+ years
• Notice Period: Immediate - 15 days

Databricks Architect

Mandatory Qualifications

• 10+ years of overall IT experience, with significant experience in Data Engineering and Data Architecture.
• Strong hands-on experience in Azure Databricks, Python, PySpark, and SQL.
• Proven experience architecting and developing reusable, metadata-driven Python frameworks.
• Strong expertise in enterprise data modeling and complex data pipeline design.
• Experience leading Data Engineering teams while remaining actively involved in hands-on development.
• Working knowledge of Pharmaceutical / Life Sciences data and business processes.
• Strong problem-solving, technical ownership, communication, and stakeholder management skills.
• Azure Databricks Architecture: Strong hands-on experience with Azure Databricks Lakehouse Architecture, Delta Lake, Unity Catalog, Databricks Workflows/Jobs, and Databricks Asset Bundles (DAB). Experience designing scalable, secure, and production-ready Databricks solutions.
• Advanced Python & PySpark: Expert-level proficiency in Python, PySpark, and SQL, with the ability to independently develop, review, troubleshoot, and optimize complex data engineering solutions. Must be comfortable contributing directly to code development.
• Reusable Python Framework Development: Proven experience architecting and developing modular, reusable, configuration-driven Python frameworks for data ingestion, transformation, validation, orchestration, and automation. Strong understanding of metadata-driven design, configurable business rules, extensible components, and reusable execution engines.
• Data Modeling & Architecture: Strong experience in conceptual, logical, and physical data modeling, relational and dimensional modeling, schema design, source-to-target mappings, CDC, SCD Type 2, and historical data processing. Ability to independently design complex data models and develop SQL DDL scripts.
• Lakehouse Data Engineering: Experience designing and implementing Bronze, Silver, and Gold data architectures, Delta tables, scalable ETL/ELT pipelines, incremental processing, and enterprise data transformation frameworks.
• Real-Time & Batch Data Processing: Hands-on experience building batch and streaming ingestion pipelines using Spark Structured Streaming, Auto Loader, Azure Event Hubs, Delta Change Data Feed (CDF), and related technologies