Senior Data Engineer - Product

CoStar Realty Information

Sydney

On-site

AUD 140,000 - 170,000

Full time

3 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Equity / stock options

Job summary

CoStar Realty Information seeks a Senior Data Engineer to design and build scalable data solutions on a Lakehouse platform, partnering with data scientists, analysts, product managers and engineers across the Domain Group. You will craft robust pipelines using DBT, Airflow, Snowflake, Databricks and AWS, and develop real-time streaming architectures with Kafka and Flink in an AWS cloud environment.

The role emphasizes modern data modelling, cost-efficient cloud data warehousing, and governance

Qualifications

  • 5+ years of hands-on commercial experience in software or data engineering.
  • 4+ years of strong development experience in SQL and Python.
  • Proven experience with data integration and orchestration tools, including Apache Airflow, DBT, and AWS Glue.
  • Expert on modern cloud data platforms and architectures, such as Snowflake, Databricks, and Lakehouse architecture
  • Experience designing and building data solutions on AWS cloud infrastructure.
  • Experience working with semi-structured and unstructured data.
  • Strong data modelling skills with the ability to evolve models in an agile environment.
  • Hands‑on experience building streaming and real-time pipelines with Apache Kafka and Apache Flink.
  • Knowledge of infrastructure-as-code tools such as CDK, CloudFormation, or Terraform.
  • Degree in computer science, engineering, mathematics, statistics, or a related technical field

Responsibilities

  • Design and build reusable, scalable data solutions on a Lakehouse platform.
  • Develop robust data pipelines using DBT, Airflow, Snowflake, Databricks and AWS services.
  • Create real-time data pipelines with Kafka and Flink for low-latency use cases.
  • Integrate data from SaaS, APIs, and streaming sources on cloud architecture.
  • Architect data solutions on AWS, Snowflake and Databricks with cost efficiency.
  • Build and optimise Lakehouse table formats (Iceberg), including partitioning and schema evolution.
  • Model data in alignment with evolving business needs in an agile environment.
  • Write and optimise SQL and Spark workloads for performance and cost efficiency.

Skills

Data engineering
SQL
Python
Data modelling
Streaming pipelines
Cloud architecture
Infrastructure as code

Education

Degree in CS/Engineering/Math/Stats

Tools

Apache Airflow
DBT
AWS Glue
Snowflake
Databricks
Kafka
Flink
CDK
CloudFormation
Terraform

Job description

Full-time On-site 2mo ago Posted 79 days ago. It may already be filled.

Agile Airflow Apache AWS CloudFormation Data Warehousing

Responsibilities
  • Solution Design: Design and build re-usable, scalable data solutions and assets on a modern Lakehouse platform across a constantly evolving landscape of products and internal data teams.
  • Pipeline Development: Build and maintain robust, reliable pipelines using DBT, Apache Airflow, Snowflake, Databricks and AWS services.
  • Streaming & Real-Time: Develop real-time data pipelines using Apache Kafka and Apache Flink to support low-latency, event-driven use cases.
  • Data Integration: Connect to a wide variety of internal and external sources via multiple methods (SaaS solutions, APIs, direct connections, streaming) on cloud architecture.
  • Cloud Data Architecture: Architect and implement data solutions on AWS, Snowflake and Databricks, applying best practices for cloud-native data warehousing and cost-efficiency.
  • Lakehouse Architecture: Build and optimise open Lakehouse table formats such as Apache Iceberg, managing schema evolution, partitioning, and performance.
  • Modern Data Modelling: Apply modern data modelling practices that evolve with the business in an agile environment.
  • Performance & Cost: Write and optimise complex SQL and Spark workloads to keep solutions efficient in both time and cost.
  • Data quality and governance
  • Cross-functional Partnership: Work closely with data scientists, analysts, product managers, and engineers across the Domain Group to deliver data solutions that drive business value.
  • Continuous Improvement: Challenge the status quo - proactively identify opportunities to improve data processes, reduce costs, and increase reliability.
  • At least 5+ years of hands-on commercial experience in software or data engineering.
  • 4+ years of strong development experience in SQL and Python.
  • Proven experience with data integration and orchestration tools, including Apache Airflow, DBT, and AWS Glue.
  • Expert on some of the modern cloud data platforms and architectures, such as Snowflake, Databricks, and Lakehouse architecture
  • Experience designing and building data solutions on AWS cloud infrastructure.
  • Experience working with semi-structured and unstructured data.
  • Strong data modelling skills with the ability to evolve models in an agile environment.
  • Hands‑on experience building streaming and real-time pipelines with Apache Kafka and Apache Flink.
  • Knowledge of infrastructure-as-code tools such as CDK, CloudFormation, or Terraform.
  • Degree in computer science, engineering, mathematics, statistics, or a related technical field
  • What's In It For You?
  • About Domain (part of the CoStar Group)
  • Equity, Diversity & Inclusion
  • Domain is enthusiastically and unapologetically committed to fostering an equitable, inclusive work culture which reflects our customers and communities. We are proactively looking for candidates from all lived experiences, including people with disability, and p

Equity / stock options

Additional Information

Senior Data Engineer - ProductJob DescriptionExciting opportunity for a Senior Data Engineer to join Domain - CoStar in a period of great opportunity and growth. You will have a direct impact on the continued success of Domain-CoStar in the APAC region. Leveraging the largest reality data sets in the local market, to enhance Domain-CoStar's product service offering and refine internal business practices.The purpose of this role is to build data platform capabilities for batch and streaming pipelines on our Lakehouse and develop data assets that serve as the source and authority for data-driven decisions and analytics used across the business. You will design, develop and operate a data platform and datapipelines which serve Domain's products and internal data teams like data product, data science, data analytics and ML/AI.

Your Match

How well this role fits your profile.

Confirmed H-1B sponsor. The Department of Labor record shows filings under COSTAR REALTY INFORMATION, INC. .

Filings 272

Approved 96.7%

Avg wage $149K

From public Department of Labor LCA disclosures. Sponsorship history is a signal about the employer, not a commitment on this role.

Company Intel
Company Intel

No data available for CoStar Realty Information yet.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data Engineer
Data Engineer

CoStar Group • Sydney

Hybrid
AUD 140,000 - 190,000
Extra annual leave days
Leadership development programs
LinkedIn Learning
+1
Senior Data Engineer - Product
Senior Data Engineer - Product

CoStar Group, Inc. • Sydney

On-site
AUD 170,000 - 210,000
Extra annual leave
Leadership development programs
LinkedIn Learning
+1
Senior Data Engineer - Product
Senior Data Engineer - Product

Domain • Sydney

On-site
AUD 140,000 - 180,000
Extra days of annual leave
Leadership development programs
LinkedIn Learning
+1
Senior Software Engineer - Audience Amplication
Senior Software Engineer - Audience Amplication

CoStar Realty Information • Sydney

On-site
AUD 120,000 - 190,000
Vision insurance
Equity / stock options
Senior Data Engineer: Lakehouse & Real-Time Pipelines
Senior Data Engineer: Lakehouse & Real-Time Pipelines

Domain • Sydney

On-site
AUD 140,000 - 180,000
Extra days of annual leave
Leadership development programs
LinkedIn Learning
+1
Senior Data Engineer: Lakehouse & Real-Time Pipelines
Senior Data Engineer: Lakehouse & Real-Time Pipelines

CoStar Realty Information • Sydney

On-site
AUD 140,000 - 170,000
Equity / stock options
Senior Data Engineer: Lakehouse, Streaming & Cloud
Senior Data Engineer: Lakehouse, Streaming & Cloud

CoStar Group • Sydney

Hybrid
AUD 140,000 - 190,000
Extra annual leave days
Leadership development programs
LinkedIn Learning
+1
Senior Data Solutions Architect: Lakehouse & Big Data
Senior Data Solutions Architect: Lakehouse & Big Data

Confidential • Sydney

On-site
AUD 180,000 - 230,000
Senior DevOps Engineer - Operations
Senior DevOps Engineer - Operations

CoStar Realty Information • Sydney

On-site
AUD 150,000 - 190,000
Vision insurance
Stock options
Senior DevOps Engineer - Operations
Senior DevOps Engineer - Operations

CoStar Group • Sydney

On-site
AUD 140,000 - 200,000
Extra days of annual leave
Leadership development programs
LinkedIn Learning
+1
Top locations