Databricks Migration Consultant

Celebal Technologies; Gurugram District, Bengaluru, Mumbai; On-site; INR 2,400,000 - 3,600,000; Full time

Job summary

Celebal Technologies is hiring 2 Senior Consultants to lead Teradata-to-Databricks migration and ETL modernization initiatives. You will own assessments, design, development, validation and production deployment for large-scale data warehouse migrations.

The role requires strong Databricks experience, PySpark development, Delta Lake and Unity Catalog knowledge. You will collaborate with architects and stakeholders to deliver scalable, governed data pipelines and optimize performance.

Qualifications

• Strong hands-on Teradata-to-Databricks migration experience is required.
• Experience with Databricks Lakehouse architecture and PySpark development.
• Knowledge of Delta Lake, Unity Catalog and data governance best practices.
• Proven ability to lead migration assessments, design, development, validation and production deployment.

Responsibilities

• Lead and execute Teradata-to-Databricks migration projects.
• Analyze Teradata objects, SQL logic, ETL workflows and data quality.
• Design migration solutions on Databricks Lakehouse and implement PySpark pipelines.
• Build scalable data pipelines with Delta Lake and optimize workloads.
• Support migration cutover, production deployment and stabilization.
• Guide junior team members and contribute to migration best practices.

Skills

Teradata-to-Databricks migration

Databricks

PySpark / Spark SQL

Delta Lake

Unity Catalog

ETL/ELT modernization

SQL & data warehousing

Data migration & validation

Job description

Role Overview

We are looking for 2 Senior Consultants with strong hands-on experience in Teradata-to-Databricks migration and ETL modernization. The ideal candidate should have experience working on large-scale data warehouse migration projects and be comfortable owning migration activities from assessment and design through development, validation and production deployment.

Key Responsibilities

• Lead and execute Teradata-to-Databricks migration projects.


• Analyze existing Teradata databases, tables, views, SQL logic, stored procedures and ETL workflows.


• Design and implement migration solutions on Databricks Lakehouse.


• Develop scalable data pipelines using PySpark and Databricks.


• Build and optimize data processing using Delta Lake.


• Implement data governance, security and access control using Unity Catalog.


• Modernize legacy ETL processes and migrate them to Databricks-based pipelines.


• Perform source-to-target mapping, data transformation and reconciliation.


• Conduct migration validation, data quality checks and performance testing.


• Optimize Databricks workloads for performance, scalability and cost.


• Support migration cutover, production deployment and post-migration stabilization.


• Collaborate with architects, business stakeholders and engineering teams to resolve migration challenges.


• Provide technical guidance to junior team members and contribute to migration best practices.

Mandatory Skills

• Strong hands-on experience in Teradata-to-Databricks migration.


• Strong Databricks experience.


• Strong PySpark / Spark SQL skills.


• Strong knowledge of Delta Lake.


• Hands-on experience with Unity Catalog.


• Experience in ETL/ELT modernization.


• Strong SQL and data warehousing concepts.


• Experience with data migration, reconciliation and validation.


• Good understanding of Lakehouse architecture.

Preferred Experience

• Experience with Teradata SQL, stored procedures and legacy ETL workloads.


• Experience with Bronze/Silver/Gold or Medallion architecture.


• Experience with Databricks Workflows / Jobs / DLT.


• Experience with CI/CD and Git-based deployment.


• Experience with Azure/AWS/GCP cloud data platforms.


• Experience in performance tuning and optimization of Spark/Databricks workloads.

Ideal Candidate

Candidates who have personally executed Teradata-to-Databricks migration projects will be strongly preferred over candidates who only have generic Databricks development experience.