Databricks Data Platform Engineer & SDET

NCS Australia; Brisbane; Presencial; AUD 120.000 - 190.000; Jornada completa

Descripción de la vacante

NCS Australia is seeking a Data Platform Engineer to advance our Databricks Lakehouse, PySpark, and SQL capabilities. This hands-on role emphasizes quality engineering, automation, and end-to-end data integrity across distributed systems.

You will validate data flows, implement CI/CD quality gates in GitLab, and collaborate with cross-functional teams to strengthen platform resilience and scalability.

Formación

• Extensive experience as a Senior Data Engineer or similar
• Hands-on with Python, PySpark, SQL in large-scale environments
• Experience with Databricks Lakehouse and data pipelines testing

Responsabilidades

• Design and maintain automated test frameworks and suites
• Build quality gates in GitLab CI/CD pipelines
• Conduct end-to-end testing of complex data flows
• Validate data integrity across Databricks, RabbitMQ, Elasticsearch
• Collaborate with developers and platform engineers on testing
• Plan load, performance, and chaos testing strategies
• Investigate data discrepancies across storage systems

Conocimientos

Python

PySpark

SQL

Databricks Lakehouse

Data Quality

Educación

Bachelor's degree in a related field

Herramientas

Databricks

Elasticsearch

RabbitMQ

GitLab CI/CD

Terraform

Descripción del empleo

NCS Australia is seeking a Data Platform Engineer to advance our Databricks Lakehouse, PySpark, and SQL capabilities. This hands-on role emphasizes quality engineering, automation, and end-to-end data integrity across distributed systems.

You will validate data flows, implement CI/CD quality gates in GitLab, and collaborate with cross-functional teams to strengthen platform resilience and scalability.