Databricks QA Automation Engineer: Data Validation & CI/CD

Vizient, Inc.; Chicago (IL); On-site; USD 89,000 - 156,000; Full time

Benefits offered by this job

Incentive eligible

Job summary

Vizient, Inc. is seeking a dedicated data test engineer to develop and maintain automated test suites for Databricks jobs, Delta tables, and ETL pipelines. You will validate complex data transformations, perform end-to-end testing, and integrate tests into CI/CD workflows within a large-scale data environment.

This role requires thorough data validation, reconciliation across systems, and robust test data creation. Vizient offers an incentive-eligible plan and comprehensive benefits.

Qualifications

• Responsibilities listed in the posting.

Responsibilities

• Develop and maintain automated test suites for Databricks jobs, Delta tables, views, data transformations, and ETL pipelines.
• Validate metric calculations, business rules, aggregations, derived values, and source-to-target data transformations within Databricks.
• Perform source-to-target reconciliation and validate data completeness, accuracy, consistency, timeliness, and referential integrity across systems.
• Develop automated tests for full and incremental loads, updates, deletes, retries, exception handling, and recovery scenarios.
• Create test data, SQL validation queries, reusable utilities, and reconciliation frameworks to improve testing efficiency and coverage.
• Integrate automated data tests into CI/CD pipelines and enhance test automation frameworks and processes.
• Perform functional, integration, regression, and end-to-end testing across large-scale data solutions.
• Analyze test failures, conduct root cause analysis, document defects, and collaborate with data engineering and platform teams to resolve issues.
• Validate data and integrations involving Stardog, knowledge graph platforms, and other data technologies.
• Test knowledge graph structures, ontologies, relationships, mappings, SPARQL queries, data lineage, and provenance.

Job description

Vizient, Inc. is seeking a dedicated data test engineer to develop and maintain automated test suites for Databricks jobs, Delta tables, and ETL pipelines. You will validate complex data transformations, perform end-to-end testing, and integrate tests into CI/CD workflows within a large-scale data environment.

This role requires thorough data validation, reconciliation across systems, and robust test data creation. Vizient offers an incentive-eligible plan and comprehensive benefits.