Data Processing Quality Engineer

Datapelago; Hyderabad; On-site; INR 1,200,000 - 1,800,000; Full time

Job summary

A leading tech firm in Hyderabad seeks an experienced professional to ensure the quality of their data processing engine. The ideal candidate will have more than 5 years of experience in data processing performance testing and a strong understanding of SQL, Python, and testing frameworks. Responsibilities include developing test strategies and executing various testing types to guarantee robust execution and performance fidelity. Join a dynamic team in a growing company focused on innovative data solutions.

Qualifications

• 5+ years of experience in data processing quality or performance testing.
• Experience on platforms like Apache Spark, Gluten, Velox, or DataFusion.
• Solid knowledge of common data processing languages.

Responsibilities

• Ensure quality of the data processing engine regarding accuracy and performance.
• Develop test strategies for complex systems.
• Conduct various levels of testing: Functional, Performance, Integration, and System Level.

Skills

Data processing quality testing

SQL

Python

Debugging

Automation

Education

BS EE/CS or equivalent

Tools

Jira

Confluence

Git

Jenkins

Job description

Ensure quality of the data processing engine, in terms of result accuracy, performance fidelity, and robust execution at scale.

Requirements

• BS EE/CS or equivalent
• 5+ years of experience in data processing quality or performance testing for database, data warehouse, or query engine applications. Experience testing for platforms such as Apache Spark, Gluten, Velox, DataFusion preferred.
• Solid knowledge of SQL, Python, and similar data processing languages
• Automation-first mindset, experienced with programming/scripting languages and automation tools.
• Strong in problem-solving and coming up with the test strategy for the complex system.
• Strong in debugging, root cause, and narrowing down the failures.
• Experience in Functional, Performance, Integration, System Level testing
• Experience with the use of public cloud platforms such as AWS, GCP, and MS Azure
• Good Knowledge of tools like Jira, Confluence, Git, Jenkins.
• Good understanding of SDLC and agile methodologies.
• Good understanding of CI/CD implementations.