Senior Data QA Engineer - Databricks & ETL

SMBC; Charlotte (NC); Hybrid; USD 97,000 - 156,000; Full time

Benefits offered by this job

Hybrid work model

Accommodations during candidacy

Job summary

SMBC is seeking a Senior QA Testing Specialist-Data Engineer in Charlotte, NC to ensure data quality across enterprise platforms built on Databricks. The role focuses on validating complex ETL pipelines, data transformations, and large datasets to enable data-driven decision-making.

The candidate will work with Data Engineers, Business Analysts, Product Owners, and Project Managers across distributed teams, developing test strategies, automation, and regression testing to ensure production

Qualifications

• 8+ years of QA testing or data engineering development roles.
• Advanced SQL (CTEs, window functions, complex joins) for large-scale data validation.
• 6+ years in test management strategy, in securities or financial services.
• 3+ years in designing process flow, business logic and UI.
• Strong experience with Azure data ecosystem and Databricks notebooks.
• Experience building data test automation and CI/CD integrations.
• Proficient in Python, PySpark, Pandas for data validation.
• Understanding ETL pipelines and data warehouse testing, including end-to-end validation.

Responsibilities

• Ensure data quality and validation before production releases.
• Define test scope, scenarios, risk-based coverage, and test strategies.
• Design and execute test cases for complex ETL pipelines; perform regression testing.
• Develop Python validation scripts to support data test automation.
• Monitor Databricks jobs; investigate pipeline failures and data quality issues.
• Log, track and manage defects using JIRA; ensure quality documentation.
• Collaborate with data engineers, architects, and business teams to resolve issues.
• Provide timely communication on quality risks, progress, and release readiness.

Skills

Advanced SQL

Test management

Python (PySpark/Pandas)

Data quality assurance

ETL/Data warehouse testing

Databricks

Git & CI/CD

Azure data ecosystem

Communication skills

Tools

Databricks notebooks

Databricks clusters

CI/CD pipelines

Git

Job description

SMBC is seeking a Senior QA Testing Specialist-Data Engineer in Charlotte, NC to ensure data quality across enterprise platforms built on Databricks. The role focuses on validating complex ETL pipelines, data transformations, and large datasets to enable data-driven decision-making.

The candidate will work with Data Engineers, Business Analysts, Product Owners, and Project Managers across distributed teams, developing test strategies, automation, and regression testing to ensure production