Test Data Support
Quality Ai
6 days ago
Be an early applicant
Application generator
Turn this role into an interview — a resume and cover letter built around what this employer wants.
Get past ATS filters
Benefits offered by this job
Job summary
QualityAI is seeking a Lakehouse Data Product Test Engineer to validate data products across Lakehouse pipelines and downstream systems. You will craft end-to-end tests, ensure data accuracy, and automate validations using Python/PySpark in a cloud-first environment.
You will collaborate with Data Engineering, Product, Platform, and Business teams to ensure data quality and release readiness, leveraging SQL and big data frameworks on AWS.
Qualifications
- Experience in data/ETL testing and data quality validation.
- Strong SQL skills and ability to write complex queries.
- Proficiency in Python or PySpark for test automation.
- Experience with big data frameworks and cloud platforms (AWS).
- Ability to test APIs and validate data services.
Responsibilities
- Design and execute data, ETL, integration, regression, and end-to-end tests for Lakehouse data products.
- Validate data ingestion, transformation, and consumption across large-scale environments.
- Develop automated data validation and testing solutions using Python/PySpark.
- Perform API testing and end-to-end validation across pipelines and downstream apps.
- Troubleshoot discrepancies and contribute to CI/CD test alignment.
Skills
Data ETL Testing
SQL
Python PySpark
Big Data
AWS
API testing
Education
Bachelor's in CS/IT
Tools
AWS EMR
Delta Lake
Databricks
Spark
Job description
Language View profile Search by Keyword Search by Location Search by Postal Code Show More Options Clear Select how often (in days) to receive an alert:
Test Data Support
Date: 29 Sept 2026 Company: QualityAI Country/Region: IN Role SummaryWe are seeking a Lakehouse Data Product Test Engineer with strong experience in data/ETL testing, big data validation, SQL, cloud platforms, API testing, and test automation. The role will focus on validating data products, ensuring data accuracy and completeness, and performing end-to-end testing across Lakehouse data pipelines and downstream applications.Key ResponsibilitiesDesign and execute data, ETL, integration, regression, and end-to-end testing for Lakehouse data products.Validate data ingestion, transformation, processing, and consumption across large-scale data environments.Perform source-to-target validation, data reconciliation, completeness, accuracy, and data quality testing.Write complex SQL queries to validate transformations, business rules, aggregations, and data integrity.Test data pipelines and workflows running on big data frameworks and cloud platforms.Develop automated data validation and testing solutions using Python/PySpark.Perform API testing to validate data services and integrations.Validate data products across multiple layers of the Lakehouse and downstream systems.Execute regression testing for new data, code, schema, and pipeline changes.Troubleshoot data discrepancies and pipeline failures and perform root-cause analysis.Build and maintain reusable test automation frameworks and regression suites.Collaborate with Data Engineering, Product, Platform, and Business teams to ensure data products meet functional and data quality requirements.Support production releases and perform post-deployment data validation.Required SkillsStrong Data/ETL Testing experienceStrong SQL skillsData validation and source-to-target reconciliationExperience with Big Data frameworksPython and/or PySparkCloud data platforms, preferably AWSExperience with AWS EMR or similar big data servicesAPI testing and integration testingTest automation and framework developmentFunctional, regression, integration, and end-to-end testingStrong troubleshooting and root-cause analysis skillsPreferred SkillsHealthcare data/domain knowledgeExperience with healthcare data standards such as HL7, FHIR, or claims/clinical dataExperience with Databricks, Spark, Delta Lake, or similar Lakehouse technologiesAWS services such as S3, EMR, Glue, Lambda, Athena, or RedshiftExperience with large-scale healthcare datasetsKnowledge of data quality and governance practicesExperience integrating automated tests into CI/CD pipelinesExperience5+ years of experience in data/ETL testing or data quality engineering, with strong hands-on experience in SQL, big data validation, Python/PySpark, cloud platforms, and test automation. Healthcare data experience is highly preferred.Top 3 Must-Have SkillsData/ETL Testing & ValidationSQL & Python/PySparkBig Data / Lakehouse & Cloud (AWS/EMR) Testing- Opens in a new tab.
- Opens in a new tab.
- Opens in a new tab.
- Opens in a new tab.
- Opens in a new tab.
'/%3e%3cpath%20d='M101.744%2012.4726L102.894%2016.1947L99.0021%2013.2201L101.744%2012.4726ZM105.3%208.40937H100.488L99.0026%203.59473L97.5116%208.41047L92.7%208.40385L96.5965%2011.3834L95.1055%2016.1942L99.0021%2013.2196L101.408%2011.3834L105.3%208.40992V8.40937Z'%20fill='white'/%3e%3cdefs%3e%3clinearGradient%20id='paint0_linear_5593_1381'%20x1='90'%20y1='9.44471'%20x2='99'%20y2='9.44471'%20gradientUnits='userSpaceOnUse'%3e%3cstop%20stop-color='%2300AD70'/%3e%3cstop%20offset='0.998594'%20stop-color='%2300B67A'/%3e%3cstop%20offset='1'%20stop-color='%23DCDCE6'/%3e%3c/linearGradient%3e%3c/defs%3e%3c/svg%3e){kind=small}
