Senior Data Engineer

Equal Experts; Sydney; Hybride; AUD 120 000 - 160 000; Plein temps

Résumé du poste

Equal Experts is seeking contract Data Engineers in Sydney to design and operate scalable data pipelines, storage and analytics solutions for blue-chip clients. You will work in cross-functional teams, applying CI/CD, TDD and XP to deliver pragmatic data capabilities while mentoring client staff.

Ideal candidates have senior experience with cloud data platforms (AWS/GCP/Azure), strong Python and SQL skills, and a proven ability to collaborate with globally distributed teams in an agile

Qualifications

• Significant experience building data pipelines and platforms.
• Experience with one or more cloud providers (AWS, GCP, Azure).
• Strong programming background in Python or similar.
• Proficient SQL and data modelling in production systems.
• Familiar with CI/CD, Git and IaC tooling.
• Ability to mentor and collaborate with distributed teams.

Responsabilités

• Determine data processing and access needs with client and team.
• Create and support highly available data pipelines and storage solutions.
• Deliver software using agile practices (pair programming, TDD, CI/CD).
• Automate data infrastructure and deployments.
• Advocate agile practices to the client organisation.
• Mentor client team members.
• Help improve the team and clients’ data capabilities.

Connaissances

Data pipelines

Python

SQL

Data modelling

CI/CD

Git

Terraform

Ansible

Mentoring

Remote collaboration

Distributed systems

Outils

AWS

GCP

Azure

Data warehouses

Description du poste

Description

Equal Experts is an innovative consultancy specialising in the delivery of custom software solutions for blue-chip enterprise and public sector clients across a range of industry sectors. We work on important projects for our clients, deliver market-leading propositions across the digital, online and mobile channels, and are recognised for our leadership in the application of agile and lean delivery methods to assure delivery. We have offices in Australia, India, South Africa, the UK, the EU and the USA.

We’re looking for contract Data Engineers with proven senior-level experience to join the Equal Experts network. Our Data Engineers are highly skilled in creating data pipelines for ingest, transformation and storage of data, usually at scale, and are familiar with standing-up analytics clusters.

This role involves using modern agile technical practices - including Continuous Integration and Deployment, TDD, and XP - to deliver timely and pragmatic solutions, as well as helping others to do their jobs in a more effective way. This role involves working in teams that use modern agile technical practices - including continuous integration and deployment and fast feedback loops - to deliver timely and pragmatic solutions, as well as helping others to do their jobs in a more effective way.

Responsibilities

• Working with the client and the team to determine data processing and access needs and how best to deliver them.
• Creation and support of highly available data pipelines and storage solutions (e.g. data warehouses).
• Delivery of software using agile (pair programming, TDD, CI/CD, etc) practices.
• Automation of data infrastructure and deployments.
• Advocating agile practices to the client organisation.
• Mentoring client team members.
• Helping improve the team and clients’ data capabilities.

Required Skills

• Significant experience working on data pipelines, platforms and projects.
• Experience in developing solutions in one or more of the main Cloud providers (AWS, GCP, Azure).
• Strong background in at least one programming language, Python or Scala preferable.
• Working with a variety of databases, data warehouses and distributed file systems in production environments.
• Strong experience in SQL and data modelling.
• Exposure with data processing at scale.
• Working with source control and infrastructure as code (Git/Github, Terraform, ansible etc.) and CI/CD tools.
• Understanding and being able to support data scientists or business intelligence teams - in data prep, mentoring and support functions.
• Strong collaboration and communication skills, including the ability to communicate effectively while working remotely with globally distributed teams.