Senior QA Data Engineer - Databricks

SMBC Group; Charlotte (NC); Hybrid; USD 93,600 - 156,000; Full time

Benefits offered by this job

Hybrid work model

Disability accommodations

Competitive benefits

Job summary

SMBC Group is seeking a Senior QA Testing Specialist- Data Engineer to ensure the accuracy and reliability of enterprise data platforms built on Databricks. You will validate ETL pipelines and data transformations, partnering with Data Engineers, Analysts, and Product Owners across global teams.

You will write advanced SQL, develop Python validation scripts, monitor Databricks jobs, and drive data quality and automation efforts in a hybrid work environment.

Qualifications

• 8+ years of experience in Quality Assurance testing or relevant Data Engineering development roles.
• Advanced SQL development using CTEs, window functions, and complex joins for large-scale data validation.
• 6+ years of experience in test management strategy; experience in securities/financial services industry.
• 3+ years of designing solutions in process flow, business logic and UI.
• Strong experience in Azure data ecosystem with Databricks notebooks and cluster management.
• Experience building data test automation frameworks with CI/CD pipelines and Git integration.
• Hands-on Python (PySpark and Pandas) for data validation and analysis.
• Thorough understanding of ETL/data warehouse testing with end-to-end validation.
• Experience with Oracle or similar enterprise financial platforms.

Responsibilities

• Validate end-to-end data using source-to-target mappings and data transformations in Databricks.
• Define test scope, scenarios, risk-based coverage, and test strategies.
• Design and execute test cases for complex ETL pipelines and perform regression testing.
• Write SQL queries and monitor Databricks jobs; investigate failures and data quality issues.
• Log, track, and manage defects in JIRA; communicate risks and release readiness.
• Collaborate with Data Engineers, Architects, and Business teams to resolve issues.

Skills

Advanced SQL

Test management

Azure Databricks

Python (PySpark)

ETL testing

Data validation

Data warehouse testing

Communication skills

Tools

Databricks notebooks

CI/CD pipelines

Git

JIRA

Oracle

Job description

SMBC Group is seeking a Senior QA Testing Specialist- Data Engineer to ensure the accuracy and reliability of enterprise data platforms built on Databricks. You will validate ETL pipelines and data transformations, partnering with Data Engineers, Analysts, and Product Owners across global teams.

You will write advanced SQL, develop Python validation scripts, monitor Databricks jobs, and drive data quality and automation efforts in a hybrid work environment.