GCP Data Engineer

Ingrity; Sydney; On-site; AUD 140,000 - 180,000; Full time

Job summary

Ingrity in Sydney is seeking a Data Engineer to design and optimize data solutions on Google Cloud Platform, building scalable pipelines with BigQuery, DataFlow, Cloud Storage, and Pub/Sub, collaborating with clients and data teams.

You will implement ETL/ELT, ensure governance and security, and contribute to CI/CD and deployment, with 5+ years of experience in data engineering. Strong SQL, Python, warehousing, and container knowledge are required. Australian work rights are essential.

Qualifications

• 5+ years of data engineering experience with GCP
• Strong SQL skills for BigQuery
• Proficient in Python for data processing
• Experience designing data warehouses and schemas
• Knowledge of containerisation is a plus
• Bachelor's degree required; master's preferred

Responsibilities

• Design, develop, and maintain scalable data pipelines on GCP.
• Implement ETL/ELT processes for ingestion and transformation.
• Optimise workflows for performance and reliability.
• Ensure data governance, security, and compliance.
• Collaborate with data scientists and analysts to enable data access.
• Maintain code repositories and CI/CD pipelines.
• Troubleshoot data infrastructure issues to minimise downtime.
• Stay up to date with industry trends and new technologies.

Skills

GCP expertise

SQL

Python

ETL/ELT design

Data modelling

CI/CD familiarity

Communication

Education

Bachelor's degree in CS/Engineering or related field

Master's degree preferred

Tools

BigQuery

DataFlow (Apache Beam)

Cloud Storage

Pub/Sub

Docker

Kubernetes

Job description

About The Role

As aData Engineer, you will play a key role in designing, developing, and optimising data solutions on Google Cloud Platform (GCP). You will work closely with clients to implement scalable and reliable data pipelines, ensuring efficient data processing and analytics.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using GCP services such as BigQuery, DataFlow (Apache Beam), and Cloud Storage.
• Implement ETL/ELT processes for data ingestion, transformation, and loading.
• Optimise data workflows for performance, scalability, and reliability.
• Ensure data governance, security, and compliance best practices.
• Collaborate with data scientists and analysts to enable seamless access to structured and unstructured data.
• Maintain code repositories and CI/CD pipelines to ensure efficient deployment processes.
• Troubleshoot and resolve data infrastructure issues, minimising downtime and performance bottlenecks.
• Stay up to date with industry trends and emerging technologies, continuously improving data engineering practices.

Key Skills & Experience

• 5+ years of experience in data engineering, with a strong background in Google Cloud Platform (GCP).
• Expert knowledge of SQL for data querying and manipulation within BigQuery.
• Experience in DataForm, Pub/Sub, BigQuery, Cloud Storage and other GCP services.
• Strong proficiency in Python for data processing tasks.
• Deep understanding and experience in data warehousing, schema design, and data modelling.
• Experience with ETL tools and frameworks for data ingestion and transformation.
• Knowledge of containerisation technologies (Docker, Kubernetes) is a plus.
• Strong problem-solving skills and ability to thrive in a fast-paced environment.
• Excellent communication and collaboration skills.
• Bachelor’s degree in Computer Science, Engineering, or a related field (Master’s preferred).

Note:

This position is based Sydney, Australia.

The candidate must have full working rights in Australia to be eligible for this role.