Lead Data Engineer: Data Platform & AI Solutions

Endava; Melbourne; Hybrid; AUD 150,000 - 190,000; Full time

Benefits offered by this job

Monthly Lifestyle Allowance

Novated Leasing

Loyalty Leave

Inclusive Parental Leave Policy

Work From Anywhere

Job summary

Endava is seeking a Lead Data Engineer to design, build and operate reliable data platforms that convert complex data into actionable business insights. You will work across the data lifecycle from architecture to reporting, using Python and SQL and modern data tools such as Databricks, Snowflake or Microsoft Fabric.

You will collaborate with clients, analysts and engineers, guiding design decisions, implementing data models, and ensuring secure, scalable data solutions.

Qualifications

• Hands-on experience delivering production data solutions using one or more of Databricks, Snowflake or Microsoft Fabric.
• Strong programming skills in Python and SQL.
• Experience designing, developing and operating data pipelines, including ETL/ELT processes, orchestration and monitoring.
• Sound understanding of data architecture and modelling techniques.
• Understanding of ontologies and semantic modelling linking data to business concepts.
• Experience optimizing data processing and troubleshooting pipeline, query and platform performance.
• Experience integrating data through APIs with authentication and error handling.
• Experience applying RBAC and data security best practices.
• Experience using Git, maintaining repositories, writing automated tests and building CI/CD workflows.
• Experience developing data visualisations and reports.
• Ability to explain technical choices clearly and work with stakeholders.

Responsibilities

• Translate business requirements into technical solutions and contribute to design decisions.
• Design, build and maintain data pipelines ingesting, transforming and delivering data from various sources.
• Develop solutions using Databricks, Snowflake or Microsoft Fabric tailored to client needs.
• Contribute to data architecture, storage, integration and modelling.
• Develop data models and semantic layers for analytics and reporting.
• Implement pipeline orchestration, scheduling, monitoring and support.
• Troubleshoot data and pipeline issues to improve performance and cost efficiency.
• Apply data quality controls and automated testing throughout the lifecycle.
• Implement RBAC and security best practices for data platforms.
• Apply DevOps practices: version control, CI/CD and automated deployment.
• Build data visualisations and reports for business users.
• Document solutions and review code to support maintainable delivery.

Skills

Databricks

Snowflake

Microsoft Fabric

Python

SQL

ETL/ELT pipelines

RBAC & data security

Git

CI/CD

Data visualization & reporting

Tools

Databricks

Snowflake

Microsoft Fabric

Job description

Endava is seeking a Lead Data Engineer to design, build and operate reliable data platforms that convert complex data into actionable business insights. You will work across the data lifecycle from architecture to reporting, using Python and SQL and modern data tools such as Databricks, Snowflake or Microsoft Fabric.

You will collaborate with clients, analysts and engineers, guiding design decisions, implementing data models, and ensuring secure, scalable data solutions.