Cloud Data Engineer: Scalable Pipelines & Ml-Ready Data

Vericent

Melbourne

Presencial

AUD 150.000 - 190.000

Jornada completa

14 días+

Consigue una respuesta de este empleador

Tu currículum y tu carta de presentación, adaptados exactamente a lo que buscan contratar.

Supera los filtros ATS

Descripción de la vacante

Vericent is partnering with a leading global technology consulting organisation to recruit an experienced Data Engineer in Melbourne. The role focuses on building modern data pipelines and cloud-based data solutions that enable analytics, reporting, and AI-driven outcomes.

You will collaborate with cross-functional teams to design, develop, and optimize scalable data integration and transformation solutions while ensuring data quality, reliability, security and performance across the data

Formación

  • Advanced SQL skills and understanding of data structures and query performance.
  • Experience with two of: Snowflake, dbt, Databricks and PySpark.
  • Strong data modelling, warehousing and analytical data patterns.
  • Experience implementing data-quality controls and reconciliation processes.
  • Strong documentation and stakeholder-communication skills.
  • Tertiary qualification in computer science, engineering, information systems or related discipline, or equivalent practical experience.

Responsabilidades

  • Design, build and maintain batch and near-real-time data pipelines.
  • Develop ingestion and transformation processes for structured and semi-structured data.
  • Build reusable data components and curated datasets for reporting, analytics and ML.
  • Develop solutions using SQL, Python, dbt, PySpark or equivalent technologies.
  • Integrate data from enterprise apps, APIs, event streams and external sources.
  • Optimise data pipelines for reliability, performance and cost.
  • Support migration and modernisation of legacy data processes.

Conocimientos

Advanced SQL
Data modelling
Data warehousing
CI/CD practices
Stakeholder communication

Educación

Tertiary qualification in computer science, engineering, information systems or related discipline

Herramientas

Snowflake
dbt
Databricks
PySpark
Git

Descripción del empleo

Data Architecture Melbourne Australia Regular / Full time

Description
About the Role

Vericent is partnering with a leading global technology consulting organisation to recruit an experienced Data Engineer. This is an exciting opportunity to work on enterprise-scale data initiatives, building modern data pipelines and cloud-based data solutions that enable analytics, reporting, and AI-driven outcomes.

In this role, you'll collaborate with cross-functional teams, including data scientists, software engineers, product owners, and business stakeholders, to design, develop, and optimise scalable data integration and transformation solutions. You'll also play a key role in ensuring data quality, reliability, security, and performance across the entire data lifecycle.

Key Responsibilities
  • Design, build and maintain batch and near-real-time data pipelines.
  • Develop ingestion and transformation processes for structured and semi-structured data.
  • Build reusable data components and curated datasets for reporting, analytics and machine learning.
  • Develop solutions using SQL, Python, dbt, PySpark or equivalent technologies.
  • Integrate data from enterprise applications, APIs, event streams and external sources.
  • Optimise data pipelines for reliability, performance and cost.
  • Support migration and modernisation of legacy data processes.
Data Modelling and Quality
  • Develop logical and physical data models aligned to business requirements.
  • Build dimensional, relational and analytical data structures.
  • Implement data-quality rules, reconciliations and validation controls.
  • Investigate data defects and coordinate remediation with source-system and product teams.
  • Maintain data definitions, lineage, metadata and technical documentation.
  • Apply appropriate handling for sensitive, customer and commercially restricted data.
Engineering and DevOps Practices
  • Develop maintainable, tested and version-controlled data solutions.
  • Implement automated testing across ingestion, transformation and data-quality processes.
  • Support CI/CD pipelines and controlled deployment across development, test and production.
  • Participate in code reviews and apply engineering standards.
  • Implement monitoring, alerting and operational logging.
  • Reduce manual deployment and support activities through automation.
Production Support and Operations
  • Monitor production pipelines, schedules and data-processing workloads.
  • Investigate and resolve failed jobs, delayed data and data-quality incidents.
  • Participate in incident, problem and change-management processes.
  • Perform root-cause analysis and implement permanent remediation.
  • Maintain operational runbooks, support procedures and recovery steps.
  • Work with Platform Operations to ensure data products meet availability and service-level requirements.
Collaboration and Delivery
  • Work with product owners and stakeholders to define data requirements and acceptance criteria.
  • Collaborate with data scientists to provide model-ready datasets and features.
  • Work with software engineers to support application and API data requirements.
  • Contribute to technical design, estimation and delivery planning.
  • Communicate data risks, dependencies and design decisions clearly.
Required Skills and Experience
  • Advanced SQL skills and strong understanding of data structures and query performance.
  • Experience with at least two of the following:
  • Snowflake
  • dbt
  • Databricks and PySpark
  • Experience building automated data-ingestion and transformation pipelines.
  • Strong understanding of data modelling, warehousing and analytical data patterns.
  • Experience implementing data-quality controls and reconciliation processes.
  • Experience with Git, code reviews and CI/CD practices.
  • Understanding of cloud security, access control and data‑protection requirements.
  • Experience supporting production data pipelines and resolving operational incidents.
  • Strong technical documentation and stakeholder-communication skills.
  • Tertiary qualification in computer science, engineering, information systems or a related discipline, or equivalent practical experience.
Key Deliverables and Success Measures
  • Reliable and scalable data pipelines and data products.
  • Achievement of agreed data availability and processing service levels.
  • Reduction in pipeline failures and recurring data-quality issues.
  • Accurate, traceable and well-documented datasets.
  • Automated testing, deployment, monitoring and recovery processes.
  • Improved pipeline performance and cloud‑cost efficiency.
  • Compliance with security, data-governance and engineering standards.
  • Positive engagement with product, platform, analytics and business stakeholders.
Consigue la evaluación confidencial y gratuita de tu currículum.

o arrastra y suelta tu archivo aquí

Similar jobs

Puestos de trabajo similares que vale la pena comparar

Cloud Data Engineer: Scalable Pipelines & Analytics
Cloud Data Engineer: Scalable Pipelines & Analytics

Vericent • Melbourne

Presencial
AUD 150.000 - 190.000
Expression Of Interest – Project Engineer I Data Centre
Expression Of Interest – Project Engineer I Data Centre

Cpb • Sydney

Híbrido
AUD 120.000 - 180.000
Salary packaging
Parental leave
Udemy access
+2
Senior Data Engineer
Senior Data Engineer

Dixon Appointments • Melbourne

Presencial
AUD 140.000 - 190.000
Data Engineer
Data Engineer

Finxl It Professional Services • Melbourne

Presencial
AUD 120.000 - 180.000
Senior Data Engineer
Senior Data Engineer

Work Initiatives • North Sydney

Presencial
AUD 150.000 - 210.000
Senior Data Engineer
Senior Data Engineer

Talent Insights Group • Sydney

Híbrido
AUD 150.000 - 210.000
Senior Data Engineer — Scalable Cloud Data Pipelines
Senior Data Engineer — Scalable Cloud Data Pipelines

Elysium Digital • Melbourne

Presencial
AUD 120.000 - 180.000
Senior Data Engineer
Senior Data Engineer

Dixon Appointments | Recruitment Agency • Melbourne

Presencial
AUD 140.000 - 190.000
Senior Data Engineer
Senior Data Engineer

Versent • Australia

Híbrido
AUD 150.000 - 190.000
Flexibility in work hours
Craft 180 learning program
Salary packaging options
+2
Senior Data Engineer
Senior Data Engineer

Elysium Digital • Melbourne

Presencial
AUD 120.000 - 180.000
Ubicaciones