Big Data QA Engineer

DhruvCRM Software Pvt Ltd; Pune District; On-site; INR 800,000 - 1,200,000; Full time

Job summary

DhruvCRM Software Pvt Ltd, located in Baner, Pune, is seeking a Big Data QA Engineer with expertise in Data Warehouse Testing and the Hadoop ecosystem. This role is vital for ensuring data quality and operational excellence.

The ideal candidate should possess strong analytical problem-solving skills, proficiency in SQL, and experience with automation testing. A Bachelor's or Master's degree in Computer Science or a related field is required.

This full-time position emphasizes collaboration with Data Engineering and Development teams to enhance the testing process.

Qualifications

• 4-8 years of experience in Data Warehouse Testing and Big Data technologies.
• Experience with Hadoop, Spark, and SQL.
• Strong analytical skills and ability to improve testing processes.

Responsibilities

• Develop and execute testing frameworks for large datasets.
• Perform end-to-end Data Warehouse and Big Data testing.
• Collaborate with Data Engineering and Development teams.

Skills

Apache Spark

Hadoop Ecosystem

Scala

SQL PL/SQL

Data Warehouse Testing

Education

Bachelor's or Master's Degree in Computer Science, Information Technology, Engineering, or a related field

Tools

Unix/Linux

Hive

Spark SQL

Job description

Location: Baner, Pune (Work from Office)
Experience: 4–8 Years
Employment Type: Full-Time
Notice Period: Immediate Joiners Preferred

About Us

DhruvCRM Software Pvt. Ltd. is a leading technology company specializing in CRM, Sales Force Automation (SFA), Distribution Management Systems (DMS), Retail Solutions, and business process automation. We help organizations streamline operations, improve sales performance, and accelerate digital transformation through innovative and scalable technology solutions. With a focus on innovation, collaboration, and customer success, DhruvCRM empowers businesses across industries to achieve sustainable growth.

Big Data QA Engineer Job Overview

DhruvCRM Software Pvt. Ltd. is looking for an experienced Big Data QA Engineer to join our growing technology team. The ideal candidate should have expertise in Data Warehouse Testing, Hadoop ecosystem, Spark, Scala, SQL validation, and large-scale data platform testing.

Key Responsibilities of a Big Data QA Engineer

• Develop and execute testing frameworks for validating data stored in Unix, Hadoop, S3, and Oracle environments.
• Perform end-to-end Data Warehouse and Big Data testing.
• Write complex SQL queries using Oracle, Hive, Impala, and Spark SQL. Conduct root cause analysis for defects and data discrepancies.
• Validate ETL/ELT processes and large-scale data pipelines.
• Implement automation and innovative testing approaches for large datasets.
• Follow Quality Assurance and Quality Control standards.
• Collaborate with Data Engineering, Product, and Development teams.
• Support production issue analysis and resolution.
• Contribute to testing process improvements and best practices.

Required Skills

Big Data Technologies

• Apache Spark
• Hadoop Ecosystem
• Scala
• Apache NiFi
• Apache Airflow

Database & Testing

• Oracle
• Database
• SQL PL/SQL
• Data Warehouse Testing
• ETL Testing

Platforms & Tools

• Unix/Linux
• Hive
• Impala
• Spark SQL

Preferred Skills

• Databricks
• Snowflake
• Cloudera
• Apache Iceberg
• AWS
• Azure
• Google Cloud Platform (GCP)
• CI/CD Pipelines
• DevOps Practices
• Test Automation Frameworks

Soft Skills

• Strong analytical and problem-solving skills
• Excellent communication skills
• Ability to work independently
• Strong ownership and accountability
• Quick learner with adaptability to new technologies

Educational Qualification

Bachelor's or Master's Degree in Computer Science, Information Technology, Engineering, or a related field.