Senior Data Engineer: Build Scalable AWS Data Pipelines

CareCone Group; Sydney; On-site; AUD 150,000 - 190,000; Full time

Job summary

CareCone Group in Sydney seeks an experienced Senior Data Engineer to design, build, and support scalable data pipelines and curated datasets on AWS, working with analytics, data science, and business stakeholders.

The role focuses on batch and near real-time processing, transforming data with SQL, Python, PySpark, and Spark, and maintaining data quality and operational documentation in a large-scale enterprise environment.

Qualifications

• 8–10 years Data Engineer experience in enterprise environments.
• Advanced SQL with performance tuning and complex joins.
• Hands-on AWS data engineering across S3, Glue, Redshift, Athena, EMR, dbt.
• Strong Python with PySpark/Spark.
• Experience with Teradata and Siebel CRM datasets.
• Knowledge of data warehousing concepts: dimensional modelling, partitioning, incremental loads, CDC.
• Experience with Airflow for workflow orchestration.
• Strong communication with stakeholders to translate requirements into data deliverables.
• Telco industry experience is highly desirable.

Responsibilities

• Design, develop, and maintain batch and near real-time data pipelines on AWS.
• Build ETL/ELT workflows using AWS Glue, S3, Redshift, Athena, EMR, dbt, and Airflow.
• Ingest data from databases, APIs, files, and event streams into raw, cleansed, and curated layers.
• Develop transformation logic using SQL, Python, PySpark, and Spark.
• Implement data quality checks, reconciliation controls, monitoring, and alerting.
• Support DataOps and DevOps practices including version control, CI/CD, testing, releases, and production support.
• Maintain technical documentation including data flows, mappings, job schedules, runbooks, and operational procedures.

Skills

8–10 years Data Engineer experience

SQL proficiency

Python programming

Data architecture & pipelines

Stakeholder communication

Tools

AWS data services

dbt

Airflow

PySpark

Spark

Teradata

Siebel CRM

Job description

CareCone Group in Sydney seeks an experienced Senior Data Engineer to design, build, and support scalable data pipelines and curated datasets on AWS, working with analytics, data science, and business stakeholders.

The role focuses on batch and near real-time processing, transforming data with SQL, Python, PySpark, and Spark, and maintaining data quality and operational documentation in a large-scale enterprise environment.