AWS Data Engineer

CareCone Group; Sydney; On-site; AUD 110,000 - 160,000; Full time

Job summary

CareCone Group is seeking an AWS Data Engineer to design, build, and maintain end-to-end data pipelines on the AWS Data Platform from Sydney. You will implement robust ETL/ELT workflows using AWS Glue, S3, Redshift, Athena, EMR, and dbt, and manage data ingestion from diverse sources into lake/warehouse layers.

You will develop transformation logic with SQL and Python/PySpark, implement data quality checks, and collaborate with data analysts and scientists to model datasets for analytics and ML.

Qualifications

• Advanced SQL skills required.
• Hands-on experience with Teradata and Siebel CRM data.
• Experience delivering pipelines in large enterprise platforms.
• Strong AWS experience: S3, Glue, Redshift, Athena, EMR, dbt.
• Python and PySpark for transformations.
• Experience with Airflow for orchestration.
• Understanding of data warehousing concepts.
• Experience with monitoring and operational support.

Responsibilities

• Design, develop, and maintain end-to-end data pipelines on AWS.
• Build ETL/ELT workflows using AWS services and dbt.
• Ingest data from databases, APIs, files, and streams into lake/warehouse layers.
• Develop transformation logic with SQL, Python, PySpark.
• Implement data quality checks, monitoring, and alerts.
• Collaborate with analysts/scientists on data models for analytics/ML.
• Contribute to DataOps/DevOps: versioning, CI/CD, testing, release, support.
• Document data flows, mappings, jobs, and runbooks.
• Optimize pipeline performance and manage orchestration/scheduling.
• Support production deployments and operations.

Skills

Advanced SQL

AWS data services

Python/PySpark

Airflow

dbt

Teradata

Siebel CRM data

Data warehousing concepts

DataOps/DevOps

Communication with stakeholders

Tools

dbt

Airflow

ETL/ELT pipelines

Job description

Role : AWS Data Engineer
Type of role - Permanent Position

Location : Sydney, Australia

Key Responsibilities

• Design, develop, and maintain end-to-end data pipelines (batch and near real-time) on AWS Data Platform
• Build and manage ETL/ELT workflows using AWS services (e.g., AWS Glue, S3, Redshift, Athena, EMR), dbt and orchestration tools such as Airflow
• Implement data ingestion patterns from diverse sources (databases, APIs, files, event streams) into lake/warehouse layers such as raw, cleansed, and curated data layers
• Develop transformation logic using SQL and Python/PySpark for cleansing, enrichment, and standardisation
• Implement robust data quality checks, reconciliation controls, and monitoring/alerting for failures and anomalies
• Collaborate with data analysts/data scientists to model datasets for analytics and machine learning consumption.
• Contribute to DataOps/DevOps practices: version control, CI/CD, automated testing, release management, and operational support.
• Produce and maintain technical documentation (data flows, mappings, job schedules, runbooks, and operational procedures)
• Optimise Data Pipeline performance and Support workflow orchestration and scheduling
• Support production deployments and operations

Required Skills & Experience

• No of years’ experience as a Data Engineer
• Advanced SQL skills
• Hands‑on experience working with Teradata and Siebel CRM data sets
• Experience delivering data pipelines in a large‑scale enterprise data platform environment
• Strong hands‑on AWS experience with common data services such as : Amazon S3, AWS Glue, Amazon Redshift, Amazon Athena, Amazon EMR and dbt
• Strong programming capability in Python and strong data transformation experience using PySpark (preferred) and/or Spark.
• Advanced SQL skills (query optimisation, complex joins, window functions, performance tuning)
• Experience with workflow orchestration tools such as Airflow
• Solid understanding of data warehousing concepts (dimensional modelling, partitioning, incremental loads, CDC concepts).
• Experience implementing monitoring, logging, alerting, and operational support processes.
• Strong communication skills and ability to work with stakeholders to translate requirements into data deliverables
• Telco Industry Experience is highly desirable