Solution Architect - Databricks

Thakral One; Maharashtra; On-site; INR 1,500,000 - 2,500,000; Full time

Job summary

Thakral One in Maharashtra is seeking an experienced Solution Architect - Databricks to join the Data delivery Team. This role focuses on designing enterprise-grade data solutions for a large-scale data modernization program in telecommunications. Candidates should have over 8 years in data engineering, with significant experience in Databricks, Azure cloud, and data migration projects. Responsibilities include architecting data pipelines, leading orchestration designs, and ensuring robust governance for data access.

Qualifications

• 8+ years of experience in data engineering and architecture.
• 4+ years of hands-on experience with Databricks technologies.
• 3+ years of experience with Azure cloud services.
• Proven experience in large-scale data migration projects.

Responsibilities

• Design data pipelines using Azure Databricks with Medallion Architecture.
• Lead design of hybrid orchestration frameworks for data ingestion.
• Define Unity Catalog governance strategies for data access.
• Architect Databricks Workflows for job scheduling.

Skills

PySpark & Spark SQL

Medallion Architecture

Data Modelling

ETL/ELT Pipelines

CDC Integration

Data Quality

Tools

Databricks (Delta Lake, Spark, Unity Catalog)

Azure cloud services

Oracle GoldenGate

Kafka

Job description

About the job Solution Architect- Databricks

Requirement: Solution Architect - Databricks

We are looking for an experienced Solution Architect - Databricks to join our Data delivery Team for a large-scale data modernization program with a leading telecommunications company. This role involves designing and architecting enterprise-grade data solutions on Azure Databricks, migrating legacy data warehousing systems to a modern lakehouse architecture.

Key Responsibilities

• Design and architect end-to-end data pipelines using Azure Databricks with Medallion Architecture (Bronze → Silver → Gold → Semantic layers)
• Lead the design of hybrid orchestration frameworks combining DLT-Lakeflow for CDC ingestion and metadata-driven frameworks for transformation layers
• Define and implement Unity Catalog governance strategies for data access, security, and lineage
• Architect Databricks Workflows for job scheduling, dependency management, and replacing legacy Control‑M orchestration
• Design SCD Type 2 implementations and complex transformation patterns for EDW migration

Technical Leadership

• Provide technical guidance on PySpark and Spark SQL best practices for large-scale data processing
• Define coding standards, modular notebook organization, and configuration-driven development approaches
• Lead technical decisions on schema evolution, data quality frameworks, and reconciliation strategies
• Evaluate and recommend approaches for surrogate key generation, batch control mechanisms, and selective reprocessing

Migration & Modernization

• Architect migration strategies from legacy systems (ODS, EDW, Informatica PowerCenter) to Azure Databricks lakehouse
• Design patterns for CDC ingestion using Oracle GoldenGate and Informatica IDR integration
• Define approaches for historical data migration and parallel run strategies
• Ensure minimal refactoring of existing SQL/PySpark code through metadata-driven frameworks

Operational Excellence

• Design solutions supporting BAU operations including manual data patching, hold/resume/skip controls, and entity-level reprocessing
• Architect Job/Cycle/Batch auditing frameworks aligned with existing operational patterns
• Define monitoring, alerting, and logging strategies across all pipeline layers
• Ensure solutions support targeted fixes without full refresh requirements

Stakeholder Engagement

• Collaborate with Amdocs and customer architecture teams on design approvals and technical alignment
• Participate in discovery sessions for EDW entities (DDS, SDS, DNF)
• Present technical solutions and trade‑off analyses to leadership and steering committees
• Work closely with delivery teams to translate designs into implementable solutions

Required Qualifications

Experience

• 8+ years of experience in data engineering and data architecture roles
• 4+ years of hands‑on experience with Databricks (Delta Lake, Spark, Unity Catalog)
• 3+ years of experience with Azure cloud services (ADLS Gen2, Azure DevOps, Azure Data Factory)
• Proven experience in large-scale data migration projects (EDW modernization preferred)

Technical Skills

Must Have

• PySpark & Spark SQL - Strong proficiency in writing optimized transformations
• Medallion Architecture - Hands‑on experience implementing Bronze/Silver/Gold patterns
• Data Modelling - Dimensional modeling, SCD implementations, surrogate key strategies
• ETL/ELT Pipelines - Experience with metadata-driven and configuration-driven frameworks
• CDC Integration - Experience with Oracle GoldenGate, Kafka, or similar CDC tools
• Data Quality - DQ frameworks, reconciliation, exception handling patterns

Good to Have

• Informatica - PowerCenter, IDR (for migration context)
• Control-M - Understanding for migration/replacement scenarios
• Terraform/IaC - Infrastructure provisioning for Databricks workspaces
• Telecommunications Domain

Certifications (Preferred)

• Databricks Certified Solution Architect Professional
• Microsoft Azure Solutions Architect Expert (AZ-305)
• Strong analytical and problem‑solving abilities
• Excellent communication skills - ability to articulate complex technical concepts to diverse audiences
• Ability to mentor and guide junior team members
• Collaborative mindset - working with different teams, and customer stakeholders