Senior QA Data Engineer - Data Quality Automation

Gradient AI; Boston (MA); Remote; USD 130,000 - 150,000; Full time

Benefits offered by this job

Stock options

Flexible schedule - work from home

Full benefits package

Job summary

Gradient AI is seeking a Senior QA Data Engineer to build automated data quality checks across our data pipeline and ensure trust in customer data powering our AI products. You will validate loads, define checks as code, and collaborate with engineering to ship pipelines with built-in quality gates.

The role requires 5+ years in QA, strong SQL/Python, experience with Soda/dbt, Airflow, and Databricks. Flexible remote work in the US is offered with a comprehensive benefits package.

Qualifications

• 5+ years of professional experience in quality assurance with a strong focus on backend automated testing.
• Hands-on experience building automated data tests with Soda, dbt, or similar frameworks.
• Strong SQL skills (Spark SQL a plus) and familiarity with Python.
• Experience with Airflow or similar orchestration tools, and with Databricks or a similar lakehouse environment (Delta Lake).
• Familiarity with version control and CI/CD.
• Solid understanding of testing principles, including functional, integration, and regression testing, applied to data pipelines.
• Excellent analytical, debugging, and root-cause skills.
• Strong verbal and written communication skills, including clear defect reports and quality summaries for technical and non-technical audiences.
• Ability to work independently and coach others on testing best practices.
• Bonus: Experience with health insurance or healthcare data, anomaly detection or data observability tools, or front-end test automation tools such as Cypress or Playwright.

Responsibilities

• Build and maintain an automated data quality suite that runs on data refreshes through Airflow, replacing manual, per-customer validation.
• Define checks as code (dbt tests, Soda, custom SQL) managed through version control and CI, including baseline checks so that new pipelines get coverage by default.
• Validate loads for completeness, sequencing, and schema before processing, and halt or quarantine failures.
• Reconcile data across Databricks layers and against downstream outputs such as pricing tools, at the portfolio, group, member, and claim level, using tolerance thresholds.
• Detect anomalies, edge cases, and unexpected changes in historical data between loads.
• Define severity levels and per-customer overrides and produce a data quality report for each customer that gates promotion to downstream consumers.
• Track quality metrics over time to retire flaky checks and close coverage gaps.
• Partner with engineers so that new pipelines ship with checks built in, and give engineers, stakeholders, and project managers timely feedback on quality issues and risk.

Skills

QA automation

dbt tests

Soda

SQL

Python

Airflow

Databricks

Version control

CI/CD

Testing principles

communication

independent work

Cypress/Playwright (bonus)

Tools

Airflow

Databricks

Delta Lake

Job description

Gradient AI is seeking a Senior QA Data Engineer to build automated data quality checks across our data pipeline and ensure trust in customer data powering our AI products. You will validate loads, define checks as code, and collaborate with engineering to ship pipelines with built-in quality gates.

The role requires 5+ years in QA, strong SQL/Python, experience with Soda/dbt, Airflow, and Databricks. Flexible remote work in the US is offered with a comprehensive benefits package.