Senior Databricks Engineer: Lakehouse & AI Migration Lead

NTT DATA, Inc.; Western Australia; On-site; AUD 150,000 - 190,000; Full time

Job summary

NTT DATA, Inc. is seeking an experienced Senior Databricks Engineer to join a high-performing delivery team on a 12-month contract.

You will design and deliver scalable data solutions using Databricks, Spark, PySpark and SQL across enterprise environments, lead migrations to modern lakehouse architectures, and shape cloud-enabled data platforms with CI/CD best practices. You will collaborate with architects, engineers and stakeholders to translate requirements into production-ready solutions

Qualifications

• Extensive commercial experience with Databricks and enterprise-grade data platforms.
• Strong hands-on expertise with Apache Spark, PySpark, Python and SQL.
• Proven experience delivering migrations from legacy data platforms and data warehouses into Databricks environments.
• Strong understanding of Delta Lake, Lakehouse Architecture, data modelling, data ingestion frameworks and transformation pipelines.
• Experience with cloud environments including Microsoft Azure, Azure Synapse Analytics, and/or AWS.
• Knowledge of CI/CD, DevOps practices, performance optimisation, automated deployment processes and modern software engineering methodologies.

Responsibilities

• Design, develop and deliver scalable data engineering solutions using Databricks, Apache Spark and PySpark across complex enterprise environments.
• Lead the migration of legacy data platforms, warehouses and workloads into modern Databricks Lakehouse architectures.
• Build and optimise high-volume data ingestion, transformation and processing pipelines using Python, SQL and Spark technologies.
• Assess existing data architectures and provide technical recommendations to support platform modernisation initiatives.
• Troubleshoot and resolve complex data, integration and performance issues while ensuring reliability, scalability and maintainability.
• Collaborate with technical and business stakeholders to translate requirements into production-ready solutions and provide technical leadership throughout delivery.

Skills

Databricks

Apache Spark

PySpark

Python

SQL

Data modelling

Data ingestion

CI/CD

DevOps

Lakehouse

Cloud platforms

Migrations

Tools

Databricks

Azure Synapse Analytics

AWS

Job description

NTT DATA, Inc. is seeking an experienced Senior Databricks Engineer to join a high-performing delivery team on a 12-month contract.

You will design and deliver scalable data solutions using Databricks, Spark, PySpark and SQL across enterprise environments, lead migrations to modern lakehouse architectures, and shape cloud-enabled data platforms with CI/CD best practices. You will collaborate with architects, engineers and stakeholders to translate requirements into production-ready solutions