Senior Data Engineer — AWS Data Pipelines & Spark Expert

CareCone Group; Sydney; On-site; AUD 140,000 - 190,000; Full time

Job summary

CareCone Group in Sydney is seeking a Senior Data Engineer to design, build, and support scalable data pipelines and curated datasets on AWS. The role involves ownership of complex data pipelines in a large-scale enterprise and collaboration with analytics, data science, and business stakeholders.

The ideal candidate will have strong SQL and Python capabilities, hands-on AWS data services experience, and the ability to deliver reliable data solutions across batch and real-time workloads.

Qualifications

• 8-10 years' experience as a Data Engineer in enterprise-scale environments.
• Advanced SQL skills including query optimisation, complex joins, window functions, and performance tuning.
• Strong hands-on AWS data engineering experience across S3, Glue, Redshift, Athena, EMR, and dbt.
• Strong Python programming skills with PySpark and/or Spark experience.
• Experience with Teradata and Siebel CRM datasets.
• Good understanding of data warehousing concepts including dimensional modelling, partitioning, incremental loads, and CDC.
• Experience with workflow orchestration tools such as Airflow.
• Strong communication skills with the ability to work closely with stakeholders and translate requirements into data deliverables.
• Telco industry experience is highly desirable.

Responsibilities

• Design, develop, and maintain batch and near real-time data pipelines on AWS.
• Build ETL/ELT workflows using AWS Glue, S3, Redshift, Athena, EMR, dbt, and Airflow.
• Ingest data from databases, APIs, files, and event streams into raw, cleansed, and curated layers.
• Develop transformation logic using SQL, Python, PySpark, and Spark.
• Implement data quality checks, reconciliation controls, monitoring, and alerting.
• Support DataOps and DevOps practices including version control, CI/CD, testing, releases, and production support.
• Maintain technical documentation including data flows, mappings, job schedules, runbooks, and operational procedures.

Skills

SQL

Python

Airflow

dbt

PySpark

Spark

Data pipelines

Communication

Tools

S3

Glue

Redshift

Athena

EMR

dbt

Airflow

Job description

CareCone Group in Sydney is seeking a Senior Data Engineer to design, build, and support scalable data pipelines and curated datasets on AWS. The role involves ownership of complex data pipelines in a large-scale enterprise and collaboration with analytics, data science, and business stakeholders.

The ideal candidate will have strong SQL and Python capabilities, hands-on AWS data services experience, and the ability to deliver reliable data solutions across batch and real-time workloads.