Big Data Manual QA Professional

Shashwath Solution; Dadri; On-site; INR 800,000 - 1,500,000; Full time

Job summary

Shashwath Solution in India seeks a QA engineer specializing in Big Data testing to validate data pipelines, ETL jobs, and data quality across Azure Data Lake, Data Factory, and Synapse. You will design test strategies, automate where possible, and collaborate with developers, analysts, and stakeholders to ensure data integrity, performance, and reliability in large-scale data platforms.

Experience with PySpark, SQL, Jira and Agile practices is expected, as is contributing to comprehensive test

Qualifications

• Experience in QA testing for Big Data environments, with emphasis on Azure Data Lake, Data Factory and Synapse.
• Proficiency in SQL for data validation and testing.
• Hands-on PySpark for data manipulation and validation.
• Experience with Databricks for big data processing and quality checks.
• Familiarity with Jira-based defect/test-case management (X-Ray or Jira Zephyr).
• Awareness of Agile methodologies and Scrum ceremonies.

Responsibilities

• Develop and execute test scripts for data pipelines, transformations, and integrations.
• Formulate test strategies including smoke, performance, functional and regression testing.
• Collaborate with development teams to assess data workflow changes and update test cases.
• Design and run tests for data validation, storage, and retrieval using Azure services.
• Enhance automated tests as new features are developed and ensure timely delivery.
• Participate in data reconciliation and verify Data Quality frameworks.
• Share knowledge and best practices with analysts and technology teams to document testing processes.
• Communicate testing progress with stakeholders and highlight blockers.
• Maintain an understanding of Azure Data Lake platform's data landscape to ensure thorough testing.

Skills

QA testing

Big Data

SQL

PySpark

Azure Data Factory

Azure Synapse Analytics

Databricks

Jira

Agile

Test automation

Data Lake

Tools

X-Ray

Jira Zephyr

Job description

Develop and execute test scripts to validate data pipelines, transformations, and integrations.

Formulate and maintain test strategies including smoke, performance, functional, and regression testing to ensure data processing and ETL jobs meet requirements.

Collaborate with development teams to assess changes in data workflows and update test cases to preserve data integrity.

Design and run tests for data validation, storage, and retrieval using Azure services like Data Lake, Synapse, and Data Factory, adhering to industry standards.

Continuously enhance automated tests as new features are developed, ensuring timely delivery per defined quality standards.

Participate in data reconciliation and verify Data Quality frameworks to maintain data accuracy, completeness, and consistency across the platform.

Share knowledge and best practices by collaborating with business analysts and technology teams to document testing processes and findings.

Communicate testing progress effectively with stakeholders, highlighting issues or blockers, and ensuring alignment with business objectives.

Maintain a comprehensive understanding of the Azure Data Lake platform's data landscape to ensure thorough testing coverage.

Skills & Experience

3-6 years of QA experience with a strong focus on Big Data testing, particularly in Data Lake environments on Azure's cloud platform.

Proficient in Azure Data Factory, Azure Synapse Analytics and Databricks for big data processing and scaled data quality checks.

Proficiency in SQL, capable of writing and optimizing both simple and complex queries for data validation and testing purposes.

Proficient in PySpark, with experience in data manipulation and transformation, and a demonstrated ability to write and execute test scripts for data processing and validation.

Hands-on experience with Functional & system integration testing in big data environments, ensuring seamless data flow and accuracy across multiple systems.

Knowledge and ability to design and execute test cases in a behaviour-driven development environment.

Fluency in Agile methodologies, with active participation in Scrum ceremonies and a strong understanding of Agile principles.

Familiarity with tools like Jira, including experience with X-Ray or Jira Zephyr for defect management and test case management.

Proven experience working on high-traffic and large-scale software products, ensuring data quality, reliability, and performance under demanding conditions.

Mandatory Key Skills

Quality Assurance,Manual Testing,Jira,PySpark,SQL,Azure Data Factory,Azure Synapse Analytics,Big Data