Data Engineer

HCLTech; Bengaluru; On-site; INR 1,000,000 - 1,500,000; Full time

Job summary

A leading technology firm in Bengaluru is looking for a Mid-Senior level Data Quality Testing professional. The ideal candidate should have 5 to 7 years of experience in Python and data pipeline QA. Responsibilities include developing test strategies, validating data integrity, and optimizing queries. This full-time role offers opportunities for growth within the IT and consulting sectors.

Qualifications

• 5 to 7 years of experience in data pipeline testing.
• Immediate availability for joining.

Responsibilities

• Develop and execute test strategies focused on data pipelines.
• Perform data integrity validation across distributed systems.
• Optimize and validate queries for performance.

Skills

Python

Pyspark

Postgres

Data Pipeline QA

Tools

SQL

Trino

Amazon Redshift

Job description

Qualifications

Primary skill: Python, Pyspark, Postgres, Data Pipeline QA

Notice Period: Immediate joiners

Experience: 5 to 7 Years

Responsibilities

• Develop and execute test strategies focused on data pipelines, including ingestion, transformation, storage, and retrieval.
• Perform data integrity validation by verifying schema consistency, data completeness, and correctness across distributed systems.
• Conduct Change Data Capture (CDC) testing to ensure seamless data updates and synchronization.
• Optimize and validate queries for performance and correctness using SQL, Trino, and Amazon Redshift.
• Implement automated data validation and regression tests to monitor for anomalies, data drift, and pipeline failures.
• Perform load testing and stress testing for large-scale data processing workflows.
• Identify bottlenecks and performance issues in ETL/ELT workflows and work with data engineers to optimize them.
• Ensure compliance with data governance standards, including security, retention, and access controls.
• Collaborate with engineers and reliability teams to define best practices for data quality and testing automation.
• Document test cases, defects, and resolutions, providing insights for continuous improvements.

Seniority level

• Mid-Senior level

Employment type

• Full-time

Job function

• Information Technology, Engineering, and Consulting

Industries

• IT Services and IT Consulting, Engineering Services, and Business Consulting and Services