Lead Data Engineer - AWS & Databricks Platform (Hybrid)

Asx; Sydney; Hybrid; AUD 140.000 - 190.000; Vollzeit

Benefits dieser Stelle

Monthly Lifestyle Allowance: Health/wb

Novated Leasing: Pre-tax car leasing

Loyalty Leave: extra leave days

Inclusive Parental Leave Policy

Work From Anywhere: 20 days/year

Zusammenfassung

Endava is seeking a Lead Data Engineer to design, build and operate reliable data platforms that transform complex data into actionable business insights. You’ll work across the data lifecycle with Python/SQL and hands-on experience in Databricks, Snowflake or Microsoft Fabric.

You’ll translate business requirements into secure, scalable data solutions, guide engineering decisions and strengthen engineering practices while collaborating with clients, analysts and data scientists.

Qualifikationen

• Hands-on experience delivering production data solutions using one or more of Databricks, Snowflake or Microsoft Fabric.
• Strong programming skills in Python and SQL.
• Experience designing, developing and operating data pipelines, including ETL/ELT processes, orchestration and monitoring.
• Sound understanding of data architecture and modelling techniques, including analytical and dimensional modelling.
• Understanding of ontologies and semantic modelling, and how they connect data structures to business concepts.
• Experience optimizing data processing and troubleshooting pipeline, query and platform performance.
• Experience integrating data through APIs, including authentication, error handling and reliable data exchange.
• Experience applying role-based access control (RBAC) and data security best practices.
• Experience using Git, maintaining repositories, writing automated tests and building CI/CD workflows for data solutions.
• Experience developing data visualisations and reports.
• Ability to explain technical choices clearly and work with stakeholders to guide engineers.

Aufgaben

• Translate business requirements into technical solutions; provide guidance and design decisions.
• Design, build and maintain data pipelines ingesting data from APIs.
• Develop solutions using Databricks, Snowflake or Microsoft Fabric.
• Contribute to data architecture and solution design.
• Develop data models, ontologies and semantic layers for consistent business definitions.
• Implement pipeline orchestration, scheduling, monitoring and operational support.
• Troubleshoot data and pipeline issues; improve performance, scalability and cost efficiency.
• Apply data quality controls and automated testing throughout the data lifecycle.
• Implement RBAC and data security best practices.
• Apply DevOps practices to data engineering, including version control, CI/CD and automated deployment.
• Build data visualisations and reports for business users.
• Document solutions and share knowledge to support maintainable delivery.

Kenntnisse

Databricks

Snowflake

Microsoft Fabric

Python

SQL

Data pipelines

Data modelling

RBAC

Git

CI/CD

Data visualisation

Communication

Tools

dbt Core

Airflow

Terraform/Terragrunt

Delta Lake

Tableau

Power BI

Jobbeschreibung

Endava is seeking a Lead Data Engineer to design, build and operate reliable data platforms that transform complex data into actionable business insights. You’ll work across the data lifecycle with Python/SQL and hands-on experience in Databricks, Snowflake or Microsoft Fabric.

You’ll translate business requirements into secure, scalable data solutions, guide engineering decisions and strengthen engineering practices while collaborating with clients, analysts and data scientists.