QA Automation Engineer for Databricks & Data Pipelines

Vizient, Inc.; Irving (TX); On-site; USD 89,000 - 156,000; Full time

Job summary

Vizient, Inc. in Irving, TX is seeking a data test engineer to develop and maintain automated test suites for Databricks jobs, Delta tables, views, ETL pipelines, and data transformations.

You will validate metrics, accuracy, and timeliness across systems, and collaborate with data engineering and platform teams. Responsibilities include creating test data and SQL validation queries, integrating tests into CI/CD pipelines, and performing functional, integration, regression, and end-to-end

Qualifications

• Develop and maintain automated test suites for Databricks jobs, Delta tables, views, data transformations, and ETL pipelines.
• Validate metric calculations, business rules, aggregations, derived values, and source-to-target data transformations within Databricks.
• Perform source-to-target reconciliation and validate data completeness, accuracy, consistency, timeliness, and referential integrity across systems.
• Develop automated tests for full and incremental loads, updates, deletes, retries, exception handling, and recovery scenarios.
• Create test data, SQL validation queries, reusable utilities, and reconciliation frameworks to improve testing efficiency and coverage.
• Integrate automated data tests into CI/CD pipelines and enhance test automation frameworks and processes.
• Perform functional, integration, regression, and end-to-end testing across large-scale data solutions.
• Analyze test failures, conduct root cause analysis, document defects, and collaborate with data engineering and platform teams to resolve issues.

Responsibilities

• Develop and maintain automated test suites for Databricks jobs, Delta tables, views, data transformations, and ETL pipelines.
• Validate metric calculations, business rules, aggregations, derived values, and source-to-target data transformations within Databricks.
• Perform source-to-target reconciliation and validate data completeness, accuracy, consistency, timeliness, and referential integrity across systems.
• Develop automated tests for full and incremental loads, updates, deletes, retries, exception handling, and recovery scenarios.
• Create test data, SQL validation queries, reusable utilities, and reconciliation frameworks to improve testing efficiency and coverage.
• Integrate automated data tests into CI/CD pipelines and enhance test automation frameworks and processes.
• Perform functional, integration, regression, and end-to-end testing across large-scale data solutions.
• Analyze test failures, conduct root cause analysis, document defects, and collaborate with data engineering and platform teams to resolve issues.

Skills

Databricks testing

SQL validation

Test automation

CI/CD integration

Root cause analysis

Tools

Stardog

SPARQL

Knowledge graphs

Job description

Vizient, Inc. in Irving, TX is seeking a data test engineer to develop and maintain automated test suites for Databricks jobs, Delta tables, views, ETL pipelines, and data transformations.

You will validate metrics, accuracy, and timeliness across systems, and collaborate with data engineering and platform teams. Responsibilities include creating test data and SQL validation queries, integrating tests into CI/CD pipelines, and performing functional, integration, regression, and end-to-end