Sr. Data Architect (Databricks)

techwave; Hyderabad; On-site; INR 3,500,000 - 7,000,000; Full time

Benefits offered by this job

Restaurant options not stated

Job summary

Techwave is seeking a Senior Data Architect to lead end-to-end Databricks lakehouse architecture and data platform modernization. You will design scalable data pipelines, governance, and AI/ML workflows, while mentoring engineers and aligning data strategy with business needs.

The role emphasizes cloud-native data solutions, security governance, and collaboration with stakeholders across IT and business units. Hyderabad, India-based opportunity with global impact.

Qualifications

• 10-15+ years of IT experience.
• Hands-on experience leading Databricks architecture and implementations.
• Experience designing enterprise data platforms on cloud (Azure/AWS/GCP).
• Proven ability to migrate large data estates and modernize cloud data platforms.
• Experience integrating enterprise systems (SAP/Oracle/Salesforce/Workday).

Responsibilities

• Lead end-to-end Databricks lakehouse architecture design.
• Architect data platforms and CI/CD pipelines for cloud modernization.
• Mentor senior engineers and drive governance and standards.
• Collaborate with stakeholders to align data strategy with business goals.

Skills

Databricks

Databricks Lakehouse

Apache Spark (PySpark)

Delta Lake

Unity Catalog

Delta Live Tables

Databricks Workflows

Databricks SQL

MLflow

Structured Streaming

Education

Bachelor's degree in CS/IT/Engineering

Master's degree preferred

Tools

Azure Data Factory

Azure Data Lake Storage Gen2

Azure Synapse Analytics

Azure Key Vault

Terraform

GitHub/DevOps

Job description

Techwave , we are always in an exercise to foster a culture of growth, and inclusivity. We ensure whoever is associated with the brand is being challenged at every step and is provided with all the necessary opportunities to excel in life. People are at the core of everything we do.

Who are we?

Techwave is a leading global IT and engineering services and solutions company revolutionizing digital transformations. We believe in enabling clients to maximize the potential and achieve a greater market with a wide array of technology services, including, but not limited to, Enterprise Resource Planning, Application Development, Analytics, Digital, and the Internet of things (IoT).

Founded in 2004, headquartered in Houston, TX, USA, Techwave leverages its expertise in Digital Transformation, Enterprise Applications, and Engineering Services to enable businesses accelerate their growth.

Plus, we're a team of dreamers and doers who are pushing the boundaries of what's possible.

And we want YOU to be a part of it.

Job Description

Solution Architecture

• Design end-to-end Lakehouse architectures using Databricks.
• Define enterprise data architecture aligned with business and technology strategies.
• Lead cloud data modernization and migration initiatives.
• Develop scalable, secure, and highly available data platforms.

Data Engineering

• Design ETL/ELT frameworks using Apache Spark and Databricks.
• Architect batch and real-time streaming data pipelines.
• Design Medallion Architecture (Bronze, Silver, Gold).
• Define data ingestion strategies for structured, semi-structured, and unstructured data.
• Implement Change Data Capture (CDC) and incremental processing.

Data Modeling

• Design enterprise data models using: Dimensional Modeling
• Star Schema
• Snowflake Schema
• Data Vault

• Build semantic models for reporting and analytics.

Databricks Platform Architecture

• Design and optimize: Delta Lake
• Unity Catalog
• Databricks Workflows
• Delta Live Tables (DLT)
• Databricks SQL
• MLflow
• Lakeflow (where applicable)

Establish workspace standards and architecture patterns.

• Define notebook, job, and code organization strategies.

Performance & Cost Optimization

• Optimize Spark jobs and SQL workloads.
• Improve partitioning, clustering, and file layouts.
• Reduce cloud infrastructure costs through autoscaling and efficient cluster sizing.
• Implement monitoring for compute utilization and query performance.

Security & Governance

• Implement Unity Catalog for centralized governance.
• Design role-based access control (RBAC).
• Configure row-level and column-level security.
• Establish metadata management, lineage, and auditing.
• Ensure compliance with enterprise security and regulatory standards.

AI & Advanced Analytics

• Architect AI/ML platforms using Databricks.
• Enable ML lifecycle management with MLflow.
• Design data pipelines for machine learning workloads.
• Support Generative AI and Retrieval-Augmented Generation (RAG) solutions by integrating Databricks with vector databases and AI services where appropriate.

DevOps & Automation

• Define CI/CD strategies using Azure DevOps or GitHub Actions.
• Implement Infrastructure as Code (Terraform preferred).
• Automate deployment of notebooks, jobs, workflows, and platform configurations.
• Establish version control and branching strategies.

Leadership & Governance

• Lead architecture reviews and design workshops.
• Define technical standards and best practices.
• Mentor Data Engineers and Technical Leads.
• Collaborate with business stakeholders, project managers, and cloud architects.
• Provide technical leadership during project delivery.

Required Technical Skills

• Databricks
• Databricks Lakehouse Platform
• Apache Spark (PySpark and Spark SQL)
• Delta Lake
• Unity Catalog
• Delta Live Tables
• Databricks Workflows
• Databricks SQL
• MLflow
• Structured Streaming

Cloud Platforms

• Microsoft Azure (preferred) Azure Data Lake Storage Gen2
• Azure Data Factory
• Azure Synapse Analytics
• Azure Key Vault
• Microsoft Entra ID
• AWS or Google Cloud experience is also valuable.

Data Engineering

• ETL/ELT Architecture
• Data Lakes
• Data Warehousing
• Data Modeling
• CDC
• Streaming Architecture
• Metadata Management

Programming

• Python
• PySpark
• SQL
• Scala (preferred)

Databases

• SQL Server
• Oracle
• PostgreSQL
• Snowflake

BI & Analytics

• Power BI
• Tableau
• Semantic Modeling

DevOps

• Git
• Azure DevOps
• GitHub
• Terraform
• CI/CD Pipelines

Preferred Experience

• 10-15+ years of IT experience.
• 5+ years of hands‑on Databricks architecture and implementation experience.
• Experience designing enterprise data platforms on Azure, AWS, or GCP.
• Proven experience in large-scale data migration and cloud modernization projects.
• Experience integrating enterprise systems such as SAP, Oracle, Salesforce, Workday, or other business applications.

Qualifications

• Bachelor's degree in Computer Science, Information Technology, Engineering, or a related field.
• Master's degree is preferred.

Preferred Certifications

• Databricks Certified Data Engineer Professional
• Databricks Certified Data Engineer Associate
• Databricks Certified Machine Learni