Staff Data Infrastructure Engineer Staff

D8 LooP; Kitchener; Hybrid; CAD 90,000 - 150,000; Full time

Job summary

Faire is a technology wholesale platform enabling local retailers worldwide. We are looking for a data platform engineer to own ingestion, reliability, and scalable data pipelines.

You will work with Airflow, Spark, Databricks, Snowflake, Kafka, and AWS/Terraform to build robust CDC and streaming architectures, while ensuring data quality and scalability for analysts and data scientists.

Qualifications

• Airflow at scale with managed ingestion like Fivetran.
• Data quality and observability with data contracts, SLAs and tools such as Anomalo or Monte Carlo.
• Idempotency, backfills, ordering, deduplication, and handling schema changes.
• Solid Python; knowledge of Kotlin/Java/Scala/Go is a plus.
• Strong SQL and data modeling to support analysts and data scientists.
• Understanding of issues with ordering, duplicates, snapshots, and schema evolution.
• Spark expertise, Databricks, and Snowflake usage against shared storage.
• Streaming experience with change data capture and Kafka.
• AWS experience shipping infrastructure with Terraform for cloud data platforms.
• CockroachDB changefeeds and MySQL binlogs feeding into Kafka, then Iceberg on S3.

Responsibilities

• Build the CDC and streaming ingestion layer to feed Iceberg tables on S3.
• Own reliability for the platform with SLOs and on-call incident reviews.
• Develop incremental loads and maintain data integrity across pipelines.

Skills

Airflow

Data quality

Idempotency

Python

SQL

Kafka

Spark

Databricks

Snowflake

CockroachDB

Iceberg

AWS

Terraform

Schema evolution

SLOs/on-call

Tools

Databricks

Snowflake

CockroachDB

Iceberg

Kafka

Terraform

S3

Job description

Faire · boards.greenhouse.io · checked today

Faire is a technology wholesale platform built on the belief that the future is local. Independent retailers around the globe collectively represent a multi-hundred-billion-dollar wholesale market that has historically been fragmented and offline. At Faire, we're using the power of tech, data, and machine learning to connect this thriving community of entrepreneurs across the globe.

Skills, with evidence

• Airflow / orchestration Experience operating Airflow at scale and working with managed ingestion like Fivetran. must have
• Experience with data quality and observability in practice, including data contracts, SLAs, and tools like Anomalo or Monte Carlo. must have
• Idempotency & backfills You'll be responsible for the hard details like ordering, deduplication, late data, schema changes, and backfills. must have
• Python Solid Python plus at least one of Kotlin, Java, Scala, or Go. must have
• SQL Strong SQL, and good instincts for how to model data so analysts and data scientists can actually use it. must have
• You know what goes wrong with ordering, duplicates, snapshots, and schema evolution because you've dealt with it. must have
• Spark Strong Spark skills, and experience running Databricks and Snowflake against shared storage. must have
• Streaming Deep experience with change data capture and streaming ingestion from operational databases through Kafka. must have
• AWS Experience shipping infrastructure on AWS with Terraform. must have · not practised here
• Terraform Experience shipping infrastructure on AWS with Terraform. must have · not practised here
• Warehousing Build the CDC and streaming ingestion layer: CockroachDB changefeeds and MySQL binlogs into Kafka, then into Iceberg tables on S3.
• Own reliability for the platform: SLOs, on-call, incident reviews, and the follow-through so the same thing doesn't break twice.
• Incremental loads Build the CDC and streaming ingestion layer: CockroachDB changefeeds and MySQL binlogs into Kafka, then into Iceberg tables on S3.
• Machine learning At Faire, we're using the power of tech, data, and machine learning to connect this thriving community of entrepreneurs across the globe. not practised here