Data Quality Engineer

Weekday AI (YC W21); Bengaluru; On-site; INR 1,000,000 - 1,600,000; Full time

Job summary

Weekday AI (YC W21) in Bengaluru, India is seeking an experienced Senior Data QA Engineer to ensure the accuracy and reliability of enterprise data pipelines, ETL processes, and data platforms. The role requires strong SQL, Python, data engineering testing, and test automation skills to validate data quality across Snowflake, Databricks, and Hive.

You will collaborate with Data Engineers, Data Analysts, Product Managers, and Engineering teams to identify issues early and ensure production-ready

Qualifications

• 5+ years of experience in data quality, data QA, ETL testing, data engineering testing, or similar role.

Responsibilities

• Validate ETL pipelines, data transformations, business logic, and data quality across Snowflake, Databricks, Hive, and other data platforms.
• Design and execute comprehensive test scenarios for data pipelines, data warehouses, and analytical datasets.
• Translate requirements into test cases covering KPIs, metrics, calculations, and business rules.
• Use advanced SQL to validate large datasets, identify anomalies, reconcile data, and investigate data-quality issues.
• Partner with Data Engineers to identify failure points and detect defects before production releases.
• Develop automated and reusable tests for data pipelines to improve coverage and reduce regression risk.
• Contribute to and enhance data test automation frameworks with focus on scalability and maintainability.
• Validate data accuracy, completeness, consistency, and integrity across source, transformation, and target systems.
• Perform regression testing and release validation for data platform changes.
• Collaborate with Data Analysts, Product Managers, Data Engineers, and Engineering teams to resolve data-quality issues.
• Support testing across batch and distributed data-processing environments.
• Use Python to develop automation scripts, validation utilities, and data-quality testing solutions.
• Integrate testing practices into CI/CD workflows to improve release quality and velocity.
• Investigate production data issues, perform root-cause analysis, and implement sustainable solutions.
• Maintain test documentation, validation standards, and reusable testing assets.
• Continuously improve data testing methodologies, automation coverage, and quality processes.

Skills

SQL

Python

Data testing

ETL testing

Test automation

CI/CD

Data quality

Communication

Education

Bachelor's degree in Computer Science, Information Technology, Engineering

Tools

Snowflake

Databricks

Hive

Spark

Airflow

Great Expectations

Deequ

AWS

Job description

Job Description:

This role is for one of the Weekdays clients

Salary range: Rs 1000000 - Rs 1600000 (ie INR 10-16 LPA)

Experience: 5+ yrs

Location: Bengaluru, Karnataka, India

Job Type: Full-time

We are looking for an experiencedSenior Data QA Engineerto ensure the accuracy, reliability, and quality of enterprise data pipelines, ETL processes, and data platforms. The role focuses on validating data transformations, business logic, KPIs, metrics, and data quality across modern platforms such asSnowflake, Databricks, and Hive.

The ideal candidate will combine strongSQL, Python, data engineering, and test automationskills with a deep understanding of data quality and validation. You will work closely with Data Engineers, Data Analysts, Product Managers, and Engineering teams to identify potential issues early and ensure reliable, production-ready data releases.

Key Responsibilities

• ValidateETL pipelines, data transformations, business logic, and data qualityacross Snowflake, Databricks, Hive, and other data platforms.
• Design and execute comprehensive test scenarios for data pipelines, data warehouses, and analytical datasets.
• Translate business and technical requirements into effective test cases coveringKPIs, metrics, calculations, and business rules.
• Use advancedSQLto validate large datasets, identify anomalies, reconcile data, and investigate data-quality issues.
• Partner with Data Engineers to identify potential failure points and proactively detect defects before production releases.
• Develop automated and reusable tests for data pipelines to improve coverage and reduce regression risk.
• Contribute to and enhance existingdata test automation frameworkswith a focus on scalability, reliability, and maintainability.
• Validate data accuracy, completeness, consistency, and integrity across source, transformation, and target systems.
• Perform regression testing and release validation for data platform changes.
• Collaborate with Data Analysts, Product Managers, Data Engineers, and Engineering teams to resolve data-quality issues.
• Support testing across batch and distributed data-processing environments.
• Use Python to develop automation scripts, validation utilities, and data-quality testing solutions.
• Integrate testing practices intoCI/CDworkflows to improve release quality and development velocity.
• Investigate production data issues, perform root-cause analysis, and help implement sustainable solutions.
• Maintain test documentation, validation standards, and reusable testing assets.
• Continuously improve data testing methodologies, automation coverage, and quality processes.

What Makes You a Great Fit

• 5+ years of experiencein data quality, data QA, ETL testing, data engineering testing, or a closely related role.
• Strong hands-on experience validatingdata pipelines, ETL processes, and data warehousesin production environments.
• Expert-levelSQLskills with experience working with very large datasets, including terabyte-scale data.
• Proven ability to identify data anomalies, inconsistencies, and quality issues through efficient SQL analysis.
• Strong experience withSnowflake, Databricks, Hive, or similar modern data platforms.
• Solid proficiency inPythonand experience developing automated tests for data pipelines.
• Good understanding ofApache Spark, Airflow, and modern data-processing workflows.
• Strong understanding of data warehousing, ETL/ELT concepts, data transformations, and data validation.
• Familiarity withCI/CD principlesand integrating automated testing into development and deployment workflows.
• Experience building or contributing to scalable and maintainable test automation frameworks.
• Knowledge ofBDD frameworks such as Behaveis an advantage.
• Experience working withAWS or other cloud platformsis desirable.
• Familiarity with data-quality frameworks such asGreat Expectations, Deequ, or similar custom solutions is an advantage.
• Strong analytical and problem-solving skills with excellent attention to detail.
• Excellent communication and collaboration skills with the ability to work effectively across technical and business teams.
• Bachelors degree inComputer Science, Information Technology, Engineering, or equivalent professional experience is preferred.
• Strong ownership mindset with the ability to proactively identify quality risks and drive issues through resolution.

Requirements: