Senior Data Engineer: PySpark, Databricks & Azure Pipelines

BrickRed Systems; Frisco (TX); On-site; USD 120,000 - 170,000; Full time

Job summary

BrickRed Systems in Frisco, TX is seeking an experienced Data Engineer to design and operate scalable, governed data pipelines in an Azure-native environment. You will work with PySpark, SQL, Azure Data Factory (ADF), Databricks, Python, and Snowflake to enable a third-party data enrichment platform.

The role focuses on building reliable, privacy-compliant pipelines, optimizing costs, and ensuring data quality with strong governance and access controls.

Qualifications

• 5+ years of Data Engineering experience.
• Hands-on with PySpark, SQL, and ADF.
• Experience with Databricks, ETL, Python, and Azure.
• Strong experience building and troubleshooting large-scale Spark pipelines.
• Experience with Snowflake for analytics and data warehousing.

Responsibilities

• Build and enhance ingestion pipelines for large-scale batch and event-driven data processing.
• Integrate data from third-party enrichment vendors and various platforms.
• Develop scalable ETL/ELT pipelines using PySpark, Databricks, Python, SQL, and ADF.
• Implement data validation, idempotency, replay/backfill, and deduplication strategies.
• Own monitoring, alerting, dashboarding, and operational readiness.
• Troubleshoot Spark failures and perform root cause analysis.

Skills

PySpark

SQL

ETL

Python

Snowflake

Spark

Databricks

Azure

Tools

Azure Data Factory (ADF)

Databricks

Snowflake

Job description

BrickRed Systems in Frisco, TX is seeking an experienced Data Engineer to design and operate scalable, governed data pipelines in an Azure-native environment. You will work with PySpark, SQL, Azure Data Factory (ADF), Databricks, Python, and Snowflake to enable a third-party data enrichment platform.

The role focuses on building reliable, privacy-compliant pipelines, optimizing costs, and ensuring data quality with strong governance and access controls.