Senior Data Engineer

ALOIS UK

Perth

On-site

AUD 140,000 - 190,000

Full time

47 hours ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

ALOIS UK is seeking a Senior Data Engineer to design and deliver secure, scalable data solutions across on-premises and cloud platforms. You will lead complex pipelines, mentor engineers and promote reusable patterns to improve reliability and cost efficiency.

You will coordinate with multiple teams to ensure data assets meet business needs and governance requirements while driving performance and scalability improvements across the data platform.

Qualifications

  • Seven+ years of experience in data engineering or data warehousing.
  • Experience designing end-to-end data pipelines.
  • Strong SQL and Python; practical Spark experience.
  • At least two years with Azure Databricks.
  • Experience delivering cloud-based analytics and reporting pipelines.
  • Familiarity with batch and near-real-time processing.
  • Ability to mentor and lead data engineers.

Responsibilities

  • Lead design and delivery of production-grade data pipelines and assets across on-premises and cloud platforms.
  • Translate business needs into practical engineering solutions.
  • Own or lead ingestion, transformation, modelling, storage and data consumption designs.
  • Ensure non-functional requirements: security, reliability, performance, scalability and usability.
  • Promote reusable data engineering patterns and practices.
  • Review designs, mappings, test plans and deployment docs.
  • Provide technical mentoring and peer review for data engineers.

Skills

SQL
Python
Spark
Azure Databricks
Data modeling
Mentoring

Education

Bachelor's degree in Computer Science

Tools

Azure Databricks
Airflow
Control-M
Spark

Job description

About The Role

We are looking for a Senior Data Engineer to lead the design and delivery of secure, scalable and cost-effective data solutions across on-premises and cloud platforms.

You will provide technical leadership across complex data engineering initiatives, designing batch and near-real-time pipelines and integrating data into trusted data assets for reporting, analytics and downstream applications. You will also help establish reusable engineering patterns, improve platform reliability and mentor other data engineers.

Key responsibilities
  • Lead the design and delivery of complex, production-grade data pipelines and data assets across Azure, Databricks and on-premises big-data platforms.
  • Translate product priorities, business outcomes and technical requirements into practical engineering solutions.
  • Own or lead solution design for ingestion, transformation, modelling, storage and data consumption.
  • Ensure solutions meet non-functional requirements, including security, reliability, performance, scalability, maintainability and usability.
  • Optimise cloud and platform usage to meet performance requirements while managing compute and operating costs.
  • Establish and promote reusable data engineering patterns, standards and practices.
  • Review technical designs, source-to-target mappings, test strategies, implementation plans and operational documentation.
  • Lead data profiling, wrangling and analysis for new and complex datasets.
  • Work with subject-matter experts to understand business rules and ensure data assets are fit for purpose.
  • Design data products appropriate to different consumption patterns, including reporting, analytics and downstream applications.
  • Coordinate planning, governance, architecture, engineering, testing and release activities across multiple teams.
  • Ensure pipelines meet agreed service levels, dependencies, monitoring and operational requirements.
  • Lead the resolution of complex production issues and drive permanent improvements.
  • Provide technical mentoring, coaching and peer review for data engineers.
  • Contribute to engineering roadmaps, delivery estimates, risk management and continuous improvement.
  • Promote secure, governed and high-quality data practices across the delivery community.
What you will bring
Essential Experience And Skills
  • Seven or more years of experience in data engineering, data warehousing or a related field.
  • Strong experience designing and delivering end-to-end data engineering solutions.
  • Advanced SQL and Python, with practical experience using Spark or equivalent distributed processing technologies.
  • At least two years of experience designing and delivering solutions with Azure Databricks.
  • Experience delivering cloud-based analytics or reporting pipelines.
  • Strong experience with batch processing and practical exposure to near-real-time or streaming solutions.
  • Experience with data profiling, source-to-target mapping, ETL/ELT development, testing, implementation and SQL optimisation.
  • Experience with orchestration tools such as Airflow, Control‑M or equivalent.
  • Experience applying CI/CD, version control, automated testing and release-management practices.
  • Experience designing solutions for structured and unstructured data, including JSON, XML and Avro.
  • Demonstrated ability to provide technical leadership, mentor engineers and make sound engineering decisions.
Desirable experience
  • Azure data services and cloud data warehouse technologies.
  • Cloudera, Hive, Impala, HDFS, HBase or Hue.
  • Kafka, Spark Streaming or other streaming frameworks.
  • REST APIs, microservices or Data-as-a-Service architectures.
  • Experience with high-volume batch or near-real-time processing.
  • Experience delivering data solutions across complex business domains.
  • Experience with financial or management-reporting processes.
  • Relevant Azure, Databricks or other industry certifications.
Qualifications

A degree in Computer Science, Mathematics, Statistics, Information Management, Finance, Economics or a related discipline is desirable. Equivalent practical experience will also be considered.

In This Role, You Will Be Expected To
  • Act as a technical lead for complex initiatives.
  • Review and challenge designs, implementation approaches and engineering decisions.
  • Establish practical and reusable engineering patterns.
  • Identify and manage delivery, operational and technical risks.
  • Support estimation, sequencing and prioritisation of engineering work.
  • Coach and mentor data engineers and contribute to capability development.
  • Work independently while escalating decisions that require broader architecture, security, commercial or governance approval.
What you will be successful in
  • Influencing stakeholders across business and technology teams.
  • Balancing delivery speed, engineering quality, reliability and cost.
  • Making pragmatic decisions in complex and changing environments.
  • Communicating technical concepts clearly to varied audiences.
  • Navigating unfamiliar domains, legacy codebases and complex data landscapes.
  • Leading delivery across multiple teams, priorities and time zones.
  • Maintaining a strong focus on data quality, security, governance and customer outcomes.
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer
Data Engineer

XPT Software • Sydney

On-site
AUD 140,000 - 190,000
Senior Data Engineer
Senior Data Engineer

DX1 • Melbourne

On-site
AUD 160,000 - 230,000
Senior Data Engineer
Senior Data Engineer

Work Initiatives • North Sydney

On-site
AUD 150,000 - 210,000
Senior Data Engineer
Senior Data Engineer

Xpt Software Australia Pty • New South Wales

On-site
AUD 140,000 - 180,000
Senior Data and Analytics Engineer
Senior Data and Analytics Engineer

SG Fleet • Sydney

On-site
AUD 150,000 - 210,000
Wellness days
Parental leave
Learning budget
Senior Data Engineer
Senior Data Engineer

Elysium Digital • Melbourne

On-site
AUD 120,000 - 180,000
Senior Data Engineer
Senior Data Engineer

Elysium Digital • Sydney

On-site
AUD 130,000 - 180,000
Data Engineer
Data Engineer

Mane Consulting • Canberra

On-site
AUD 110,000 - 150,000
Principal Data Engineer - Cloud, Pre-Sales & Solutioning
Principal Data Engineer - Cloud, Pre-Sales & Solutioning

TalentReady • Sydney

On-site
AUD 180,000 - 240,000
Data Engineer
Data Engineer

XPT Software Australia Pty Ltd • Sydney

On-site
AUD 180,000 - 230,000
Top locations