Senior Databricks Data Quality Engineer

Insight Global; New York (NY); On-site; USD 140,000 - 170,000; Full time

Job summary

Insight Global is seeking a Sr Data Quality Engineer, Data Lakehouse to own data quality across analytics, reporting, and marketing data within a Databricks environment. You will design automated data quality frameworks and monitoring for Bronze, Silver, and Gold layers, partnering with engineers and analysts.

In this role you’ll translate business requirements into validation rules, perform root cause analysis, and help shape the data quality roadmap while building scalable observability

Qualifications

• 5+ years in Data Engineering, Data Quality, QA Automation, or Data Observability.
• Expert-level Databricks, SQL, and Python/PySpark.
• Experience building automated data quality, testing, and monitoring solutions.
• Experience supporting Bronze/Silver/Gold data pipelines and conducting root‑cause analysis.
• Strategic, proactive mindset with strong ownership and stakeholder partnership skills.
• Delta Live Tables (DLT) experience including expectations and constraints.
• Databricks Unity Catalog experience for governance, lineage, and monitoring.
• Great Expectations, Deequ, dbt, Monte Carlo, or other data quality/observability tools.
• Experience with Airflow, CI/CD, and Git‑based deployment processes.
• Marketing, social media, email, social listening, or MarTech data experience.
• Agile environment experience.

Skills

Databricks

SQL

Python/PySpark

Data Quality/Observability

Data pipelines (Bronze/Silver/Gold)

Root-cause analysis

Stakeholder partnership

Delta Live Tables

Unity Catalog

Great Expectations/Deequ/dbt

Airflow/CI/CD/Git

MarTech data

Agile

Tools

Delta Live Tables

Unity Catalog

Great Expectations

Deequ

dbt

Monte Carlo

Job description

Insight Global is seeking a Sr Data Quality Engineer, Data Lakehouse to own data quality across analytics, reporting, and marketing data within a Databricks environment. You will design automated data quality frameworks and monitoring for Bronze, Silver, and Gold layers, partnering with engineers and analysts.

In this role you’ll translate business requirements into validation rules, perform root cause analysis, and help shape the data quality roadmap while building scalable observability