Senior Data Engineer: Databricks Lakehouse Leader

Tata Consultancy Services; Irvine (CA); On-site; USD 120,000 - 180,000; Full time

Job summary

Tata Consultancy Services seeks a hands-on Senior Data Engineer to design and operate enterprise-scale data solutions on the Databricks Lakehouse Platform. You will build batch and streaming pipelines, modernize ETL workloads, and deliver trusted data products for analytics and regulatory use cases.

The role emphasizes Databricks, Spark, PySpark, SQL, Python, dbt, Airflow, Delta Lake, Unity Catalog, and cloud data services across AWS/Azure/GCP.

Qualifications

• Typically 7–10 years of data engineering or distributed data processing experience.
• Deep hands-on experience with Databricks, Spark, PySpark, Delta Lake, Python and SQL.
• Experience building production-grade ETL/ELT pipelines for large datasets and dbt.

Responsibilities

• Design, build, test, deploy, and maintain scalable ETL/ELT pipelines on Databricks.
• Develop ingestion and transformation frameworks for structured and streaming data.
• Create and maintain Delta Lake tables with medallion architecture across layers.
• Develop dbt models, tests, and documentation; build Airflow DAGs and Databricks Workflows.

Skills

Strong troubleshooting

Effective communication

Collaboration

Tools

Databricks

Apache Spark

PySpark

Delta Lake

Python

SQL

dbt

Apache Airflow

Unity Catalog

Cloud data services (AWS/Azure/GCP)

Git

CI/CD

Databricks Workflows

Job description

Tata Consultancy Services seeks a hands-on Senior Data Engineer to design and operate enterprise-scale data solutions on the Databricks Lakehouse Platform. You will build batch and streaming pipelines, modernize ETL workloads, and deliver trusted data products for analytics and regulatory use cases.

The role emphasizes Databricks, Spark, PySpark, SQL, Python, dbt, Airflow, Delta Lake, Unity Catalog, and cloud data services across AWS/Azure/GCP.