Senior Data Engineer — Lakehouse & Real-Time Data (APAC)

Costar Group; Sydney; On-site; AUD 140,000 - 190,000; Full time

Benefits offered by this job

Extra leave days

Leadership development programs

LinkedIn Learning

Perkbox discounts

Job summary

Domain – CoStar is recruiting a Senior Data Engineer to build data platform capabilities for batch and streaming pipelines on our Lakehouse. You will develop data assets that act as the source and authority for data-driven decisions across APAC, collaborating with data science, analytics and product teams.

You will design, develop and operate data solutions on AWS, Snowflake and Databricks, applying modern data modelling and governance to improve performance and cost efficiency.

Qualifications

• 5+ years of hands-on data/software engineering experience.
• 4+ years in SQL and Python development.
• Experience with data integration/orchestration tools such as Airflow, DBT and AWS Glue.
• Proven cloud data platform experience (Snowflake, Databricks) and Lakehouse architecture.
• Experience with AWS and handling semi-structured data.

Responsibilities

• Design and build reusable, scalable data assets on a Lakehouse platform.
• Develop robust pipelines using DBT, Airflow, Snowflake, Databricks and AWS services.
• Create real-time data pipelines with Kafka and Flink for low-latency use cases.
• Connect to diverse data sources via APIs, SaaS, streaming or direct connections.
• Apply modern data modelling and governance in an agile environment.

Skills

SQL & Python

Data Modeling

Cloud Platforms

Streaming Data

Cross-functional Collaboration

Education

Degree in Computer Science or related field

Tools

Apache Airflow

DBT

AWS Glue

Snowflake

Databricks

Apache Kafka

Apache Flink

CDK

CloudFormation

Terraform

Job description

Domain – CoStar is recruiting a Senior Data Engineer to build data platform capabilities for batch and streaming pipelines on our Lakehouse. You will develop data assets that act as the source and authority for data-driven decisions across APAC, collaborating with data science, analytics and product teams.

You will design, develop and operate data solutions on AWS, Snowflake and Databricks, applying modern data modelling and governance to improve performance and cost efficiency.