Senior Data Engineer - Snowflake & Databricks Platform Lead

BrickRed Systems LLC; Seattle (WA); On-site; USD 150,000 - 230,000; Full time

Job summary

BrickRed Systems is seeking a Senior Data Engineer to design, develop, and optimize enterprise-scale data pipelines and platforms supporting Finance and Data Intelligence initiatives. You will leverage Snowflake, Databricks, Delta Lake, DLT, Unity Catalog, dbt, and PySpark to deliver scalable, secure data solutions.

You will build ELT/ETL pipelines for billing, GL, OPEX, and more, implement real-time processing with Kafka and Spark, and lead platform standards, CI/CD, and governance across Azure

Qualifications

• 7+ years experience with Snowflake and Delta Lake.
• 7+ years experience with Databricks, Delta Live Tables (DLT), and Unity Catalog.
• 5+ years experience with dbt and PySpark.
• Strong SQL and Python for data pipeline development.
• Experience with ETL/ELT, incremental ingestion, CDC and fault-tolerant processing.
• Knowledge of Azure or AWS cloud data platforms.
• Experience with data quality, testing, CI/CD, security, and governance.

Responsibilities

• Design and develop scalable ELT/ETL pipelines for finance and revenue data.
• Build batch, incremental, and streaming ingestion pipelines with CDC and event-driven patterns.
• Ensure pipeline reliability with idempotent design, error handling, and retries.
• Lead development and optimization of Snowflake and Databricks platforms (Snowpipe, streams, tasks, DLT, Unity Catalog).
• Develop modular data transformations using dbt and PySpark.
• Implement cloud-native data platforms on Azure or AWS with IaC and cost optimization.
• Develop real-time processing solutions with Kafka, Event Hub, and Spark Structured Streaming.
• Establish data quality, validation, monitoring, testing, and observability standards.
• Implement CI/CD, version control, and automated deployment workflows.
• Ensure data security, governance, lineage, auditability, and RBAC/PII controls.
• Collaborate with architects, product managers, analysts, and stakeholders to deliver reliable data solutions.
• Provide technical leadership, mentor engineers, and drive platform standards.

Skills

Snowflake

Delta Lake

Databricks

Delta Live Tables

Unity Catalog

dbt

PySpark

SQL

Python

Data modeling

ETL/ELT

CDC

Cloud platforms

CI/CD

Security & governance

Tools

Airflow

Azure Data Factory

Terraform

Job description

BrickRed Systems is seeking a Senior Data Engineer to design, develop, and optimize enterprise-scale data pipelines and platforms supporting Finance and Data Intelligence initiatives. You will leverage Snowflake, Databricks, Delta Lake, DLT, Unity Catalog, dbt, and PySpark to deliver scalable, secure data solutions.

You will build ELT/ETL pipelines for billing, GL, OPEX, and more, implement real-time processing with Kafka and Spark, and lead platform standards, CI/CD, and governance across Azure