INTL Senior Databricks QA Engineer

Insight Global; New York (NY); On-site; USD 140,000 - 170,000; Full time

Job summary

Insight Global is seeking a Sr Data Quality Engineer, Data Lakehouse to own data quality across analytics, reporting, and marketing data within a Databricks environment. You will design automated data quality frameworks and monitoring for Bronze, Silver, and Gold layers, partnering with engineers and analysts.

In this role you’ll translate business requirements into validation rules, perform root cause analysis, and help shape the data quality roadmap while building scalable observability

Qualifications

• 5+ years in Data Engineering, Data Quality, QA Automation, or Data Observability.
• Expert-level Databricks, SQL, and Python/PySpark.
• Experience building automated data quality, testing, and monitoring solutions.
• Experience supporting Bronze/Silver/Gold data pipelines and conducting root‑cause analysis.
• Strategic, proactive mindset with strong ownership and stakeholder partnership skills.
• Delta Live Tables (DLT) experience including expectations and constraints.
• Databricks Unity Catalog experience for governance, lineage, and monitoring.
• Great Expectations, Deequ, dbt, Monte Carlo, or other data quality/observability tools.
• Experience with Airflow, CI/CD, and Git‑based deployment processes.
• Marketing, social media, email, social listening, or MarTech data experience.
• Agile environment experience.

Skills

Databricks

SQL

Python/PySpark

Data Quality/Observability

Data pipelines (Bronze/Silver/Gold)

Root-cause analysis

Stakeholder partnership

Delta Live Tables

Unity Catalog

Great Expectations/Deequ/dbt

Airflow/CI/CD/Git

MarTech data

Agile

Tools

Delta Live Tables

Unity Catalog

Great Expectations

Deequ

dbt

Monte Carlo

Job description

Job Description

Insight Global is seeking a Sr Data Quality Engineer, Data Lakehouse for a top healthcare client. This individual will serve as a key member of the Data Lakehouse team, taking ownership of the quality, reliability, and trustworthiness of data used for analytics, reporting, data science, and marketing automation. Working within a Databricks environment, the engineer will design and implement automated data quality frameworks across Bronze, Silver, and Gold data layers, build monitoring and observability solutions, and proactively identify issues such as data delays, schema drift, formatting inconsistencies, and accuracy concerns before they impact downstream users. The role will partner closely with Data Engineers and Analysts to translate business requirements into technical data validation rules, conduct root cause analysis, and drive long-term improvements to the organization's data ecosystem. Supporting marketing and communications data sourced from social media, email, and social listening platforms, this individual will split their time between strategic planning and hands‑on engineering, helping define the roadmap for data quality while actively building and maintaining the solutions that ensure data integrity and business trust

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment regardless of their race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or recruiting process, please send a request to [e-mail] learn more about how we collect, keep, and process your private information, please review Insight Global's Workforce Privacy Policy: https://insightglobal.com/workforce-privacy-policy/.

Skills and Requirements

• 5+ years in Data Engineering, Data Quality, QA Automation, or Data Observability.
• Expert-level Databricks, SQL, and Python/PySpark.
• Experience building automated data quality, testing, and monitoring solutions.
• Experience supporting Bronze/Silver/Gold data pipelines and conducting root‑cause analysis.
• Strategic, proactive mindset with strong ownership and stakeholder partnership skills.
• Delta Live Tables (DLT) experience including expectations and constraints.
• Databricks Unity Catalog experience for governance, lineage, and monitoring.
• Great Expectations, Deequ, dbt, Monte Carlo, or other data quality/observability tools.
• Experience with Airflow, CI/CD, and Git‑based deployment processes.
• Marketing, social media, email, social listening, or MarTech data experience.
• Agile environment experience.