Senior Databricks Data Engineer - Lakehouse PySpark Expert

Tata Consultancy Services Limited; Brussel; On-site; EUR 90,000 - 150,000; Full time

Job summary

Tata Consultancy Services Limited is seeking an experienced Senior Databricks Data Engineer to join our team in Belgium. The role focuses on building scalable data pipelines using Databricks, PySpark, Python, Spark SQL, Lakeflow Jobs, and Unity Catalog to deliver reliable lakehouse solutions.

You will design ETL/ELT workflows, optimize workloads for performance and cost, ensure data quality, and collaborate with stakeholders across the organization. English plus Dutch or French is preferred.

Qualifications

• 10+ years of experience in the data domain, especially Data Warehousing and ETL.
• 0
• 4+ years hands-on Databricks experience.
• Strong expertise in PySpark, Spark SQL, and Python.
• Solid understanding of Databricks architecture, Lakeflow Jobs, Unity Catalog, and data modelling.
• Hands-on experience with Git.
• Strong knowledge of Data Lakes, Data Warehousing, and Lakehouse architecture.
• Databricks Certified Data Engineer Associate or Professional is an advantage.
• Excellent communication and stakeholder management skills.
• Fluent in English plus Dutch or French, or both.

Responsibilities

• Design, develop, and optimize scalable data pipelines using Databricks, Python, PySpark, and Spark SQL.
• Orchestrate and schedule workflows using Lakeflow Jobs, notebooks, and Python wheel packages.
• Work with Unity Catalog, managed tables, and streaming jobs.
• Develop robust ETL/ELT pipelines for data ingestion, transformation, and loading.
• Optimize Databricks workloads for performance, scalability, and cost efficiency.
• Ensure strong data quality, integrity, reliability, and availability.
• Troubleshoot data-related issues and continuously improve data engineering solutions.

Skills

Databricks

PySpark

Python

Spark SQL

Lakeflow Jobs

Unity Catalog

Git

Tools

Databricks

PySpark

Python

Spark SQL

Lakeflow

Unity Catalog

Job description

Tata Consultancy Services Limited is seeking an experienced Senior Databricks Data Engineer to join our team in Belgium. The role focuses on building scalable data pipelines using Databricks, PySpark, Python, Spark SQL, Lakeflow Jobs, and Unity Catalog to deliver reliable lakehouse solutions.

You will design ETL/ELT workflows, optimize workloads for performance and cost, ensure data quality, and collaborate with stakeholders across the organization. English plus Dutch or French is preferred.