Data QA Engineer | Rayosys | Remote

Rayosys; United States; Remote; USD 105,000 - 145,000; Full time

Benefits offered by this job

Remote work

Open to international candidates

Job summary

Rayosys seeks an experienced Data QA Engineer to own validation, ETL testing, and data reconciliation across Azure Databricks pipelines in a remote setup.

You will design and execute testing frameworks, write SQL and Python tests, and ensure data integrity from ingestion to reporting in large-scale analytics environments.

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, Software Engineering, Data Science or a related field.
• 5+ years of professional experience in data quality assurance, ETL testing, or database validation.
• Hands-on expertise with Azure Databricks and cloud-native big data architectures.
• Experience in ETL testing, data migration validation, and automated data reconciliation.
• Strong SQL skills for relational and distributed data stores.
• Proficiency in Python for data testing, scripting, and test automation frameworks.
• Ability to analyze large datasets and detect subtle data anomalies or regressions.
• Excellent problem-solving, analytical, and remote cross-functional communication skills.

Responsibilities

• Design, develop, and execute data QA strategies, test plans, and test cases for complex ETL pipelines.
• Validate big data transformations and processing workflows in Azure Databricks.
• Perform data reconciliation, quality assessments, and schema verification across enterprise databases.
• Write advanced SQL queries and Python scripts for automated data validation and comparisons.
• Test, verify, and monitor batch and streaming data ingestion pipelines, data lakes, and warehouses.
• Log, track, and troubleshoot data discrepancies, pipeline failures, and performance bottlenecks.
• Collaborate with data engineers, software developers, and product managers in an Agile environment.
• Maintain thorough documentation of test scenarios, bug reports, data lineage verifications, and quality metrics.
• Ensure adherence to data governance, security baselines, and quality benchmarks.

Skills

Azure Databricks

SQL

Python

ETL testing

Data QA

Data governance

Data reconciliation

Cloud-native big data

Education

Bachelor’s degree in CS/IT/SE/Data Science

Tools

Azure Databricks

Apache Airflow

Azure DevOps

GitHub Actions

Microsoft Purview

Job description

Position Summary:

Rayosys is seeking an experienced, highly technical, and meticulous Data QA Engineer with 5+ years of professional quality assurance experience to join our remote technology team. In this specialized data engineering and quality assurance role, you will spearhead comprehensive validation, ETL testing, and data reconciliation across advanced Azure Databricks analytics pipelines and distributed big data environments. You will work closely with cross-functional data engineers, software architects, and product teams to guarantee absolute data integrity, performance, and accuracy across large-scale enterprise data platforms. Ideal candidates bring a robust academic foundation in computer science, deep mastery of SQL and Python, and a proven track record of testing complex cloud-native data lakes and ETL workflows.

Detailed Job Description:

As a Data QA Engineer at Rayosys, you will take full ownership of designing, implementing, and executing rigorous testing frameworks for enterprise data warehousing and analytics pipelines. Your day-to-day responsibilities encompass validating ETL data transformations, conducting deep SQL queries and data reconciliation checks, and writing automated Python test scripts to ensure seamless data quality from ingestion to reporting. You will evaluate complex big data workflows running on Azure Databricks, identify data anomalies, and collaborate with engineering squads to resolve pipeline bottlenecks. Working in a collaborative remote environment, you will maintain comprehensive test documentation, verify data accuracy against business requirements, and uphold enterprise data governance standards.

Key Responsibilities:

• Design, develop, and execute comprehensive data QA strategies, test plans, and test cases for complex ETL pipelines.
• Validate big data transformations and data processing workflows within Azure Databricks environments.
• Conduct rigorous data reconciliation, data quality assessments, and schema verification across enterprise databases.
• Write advanced SQL queries and Python scripts for automated data validation and data comparison testing.
• Test, verify, and monitor batch and streaming data ingestion pipelines, data lakes, and data warehouses.
• Identify, log, track, and troubleshoot data discrepancies, pipeline failures, and performance bottlenecks.
• Collaborate closely with data engineers, software developers, and product managers in an Agile development cycle.
• Maintain meticulous documentation of test scenarios, bug reports, data lineage verifications, and quality metrics.
• Ensure strict adherence to data governance, security baselines, and enterprise data quality benchmarks.

Required Qualifications & Skills:

• Bachelor’s degree in Computer Science, Information Technology, Software Engineering, Data Science, or a related field.
• 5+ years of professional experience in data quality assurance, ETL testing, or database validation.
• Strong hands-on technical expertise with Azure Databricks and cloud-native big data architectures.
• Extensive practical experience in ETL testing, data migration validation, and automated data reconciliation.
• Deep proficiency in writing complex SQL queries for relational databases and distributed data stores.
• Strong programming proficiency in Python for data testing, scripting, and test automation frameworks.
• Proven ability to analyze large datasets and identify subtle data anomalies or structural regressions.
• Exceptional problem-solving, analytical, and remote cross-functional communication skills.

Nice-to-Have Skills:

• Professional certifications such as Databricks Certified Data Engineer Associate/Professional or ISTQB Advanced Test Analyst.
• Experience with orchestration and CI/CD tools including Apache Airflow, Azure DevOps, and GitHub Actions.
• Familiarity with data governance and cataloging tools such as Microsoft Purview, Collibra, or Apache Atlas.
• Prior working experience in financial services, healthcare, telecommunications, or enterprise SaaS data teams.
• Contributions to open-source data testing libraries or active participation in data engineering communities.

Application Information:

• Recruiter: Rayosys Recruitment Team
• Contact Name: Alekh Bisen / HR Team
• Email: [e-mail] / [e-mail]
• Phone: Unspecified
• Application URL: Unspecified
• Salary/Rate: Competitive and commensurate with experience
• Deadline: Open until filled
• Notice Period: Immediate to short notice preferred
• Contract Duration: Permanent / Full-Time (Remote)

Recruitment Pro Tip:

When applying for senior data QA roles, ensure your resume explicitly highlights your hands‑on experience with Azure Databricks, data reconciliation, and Python‑based ETL testing frameworks.