Senior Data Engineer - Product

Costar Group; Sydney; On-site; AUD 140,000 - 190,000; Full time

Benefits offered by this job

Extra leave days

Leadership development programs

LinkedIn Learning

Perkbox discounts

Job summary

Domain – CoStar is recruiting a Senior Data Engineer to build data platform capabilities for batch and streaming pipelines on our Lakehouse. You will develop data assets that act as the source and authority for data-driven decisions across APAC, collaborating with data science, analytics and product teams.

You will design, develop and operate data solutions on AWS, Snowflake and Databricks, applying modern data modelling and governance to improve performance and cost efficiency.

Qualifications

• 5+ years of hands-on data/software engineering experience.
• 4+ years in SQL and Python development.
• Experience with data integration/orchestration tools such as Airflow, DBT and AWS Glue.
• Proven cloud data platform experience (Snowflake, Databricks) and Lakehouse architecture.
• Experience with AWS and handling semi-structured data.

Responsibilities

• Design and build reusable, scalable data assets on a Lakehouse platform.
• Develop robust pipelines using DBT, Airflow, Snowflake, Databricks and AWS services.
• Create real-time data pipelines with Kafka and Flink for low-latency use cases.
• Connect to diverse data sources via APIs, SaaS, streaming or direct connections.
• Apply modern data modelling and governance in an agile environment.

Skills

SQL & Python

Data Modeling

Cloud Platforms

Streaming Data

Cross-functional Collaboration

Education

Degree in Computer Science or related field

Tools

Apache Airflow

DBT

AWS Glue

Snowflake

Databricks

Apache Kafka

Apache Flink

CDK

CloudFormation

Terraform

Job description

Senior Data Engineer - Product

Job Description

Exciting opportunity for a Senior Data Engineer to join Domain – CoStar in a period of great opportunity and growth. You will have a direct impact on the continued success of Domain-CoStar in the APAC region. Leveraging the largest reality data sets in the local market, to enhance Domain-CoStar’s product service offering and refine internal business practices.

The purpose of this role is to build data platform capabilities for batch and streaming pipelines on our Lakehouse and develop data assets that serve as the source and authority for data-driven decisions and analytics used across the business. You will design, develop and operate a data platform and data

pipelines which serve Domain’s products and internal data teams like data product, data science, data analytics and ML/AI.

What you’ll do

• Solution Design: Design and build re-usable, scalable data solutions and assets on a modern Lakehouse platform across a constantly evolving landscape of products and internal data teams.
• Pipeline Development: Build and maintain robust, reliable pipelines using DBT, Apache Airflow, Snowflake, Databricks and AWS services.
• Streaming & Real-Time: Develop real-time data pipelines using Apache Kafka and Apache Flink to support low-latency, event-driven use cases.
• Data Integration: Connect to a wide variety of internal and external sources via multiple methods (SaaS solutions, APIs, direct connections, streaming) on cloud architecture.
• Cloud Data Architecture: Architect and implement data solutions on AWS, Snowflake and Databricks, applying best practices for cloud-native data warehousing and cost-efficiency.
• Lakehouse Architecture: Build and optimise open Lakehouse table formats such as Apache Iceberg, managing schema evolution, partitioning, and performance.
• Modern Data Modelling: Apply modern data modelling practices that evolve with the business in an agile environment.
• Performance & Cost: Write and optimise complex SQL and Spark workloads to keep solutions efficient in both time and cost.
• Data quality and governance
• Cross-functional Partnership: Work closely with data scientists, analysts, product managers, and engineers across the Domain Group to deliver data solutions that drive business value.
• Continuous Improvement: Challenge the status quo — proactively identify opportunities to improve data processes, reduce costs, and increase reliability.

Essential:

• At least 5+ years of hands-on commercial experience in software or data engineering.
• 4+ years of strong development experience in SQL and Python.
• Proven experience with data integration and orchestration tools, including Apache Airflow, DBT, and AWS Glue.
• Expert on some of the modern cloud data platforms and architectures, such as Snowflake, Databricks, and Lakehouse architecture
• Experience designing and building data solutions on AWS cloud infrastructure.
• Experience working with semi-structured and unstructured data.
• Strong data modelling skills with the ability to evolve models in an agile environment.
• Hands‑on experience building streaming and real‑time pipelines with Apache Kafka and Apache Flink.
• Knowledge of infrastructure‑as‑code tools such as CDK, CloudFormation, or Terraform.
• Degree in computer science, engineering, mathematics, statistics, or a related technical field

What’s In It For You?

Working at CoStar Group means you'll enjoy a culture of collaboration and innovation that attracts the best and brightest across a broad range of disciplines. We offer you generous compensation and performance‑based incentives. As well as having an outstanding working environment, other perks include extra days of annual leave, leadership development programs, LinkedIn Learning, Perkbox discounts and much more!