QA Lead - Big Data & Automation

Tata Consultancy Services; Dallas (TX); On-site; USD 100,000 - 110,000; Full time

Benefits offered by this job

Discretionary Annual Incentive

Comprehensive Medical Coverage

Family Support: Leaves

Insurance Options

Commuter Benefits

Certification & Training Reimbursement

Time Off: Vacation & Holidays

Legal & Financial Assistance

Job summary

Tata Consultancy Services seeks a Quality Assurance Lead (Contractor) to drive end-to-end testing strategy for large-scale data platforms. You will lead QA efforts across batch and streaming environments, focusing on data integrity, regulatory compliance, and system reliability.

The role requires strong Python/PySpark automation, experience with Hadoop ecosystems, and expertise in CI/CD integration with Jenkins and Git.

Qualifications

• 8+ years of experience in Quality Assurance / Test Engineering with leadership responsibilities.
• Strong proficiency in Python and PySpark for automation and data validation.
• Solid experience with big data platforms (Hadoop, Hive, Impala, Sqoop).
• Hands-on expertise in Oracle PL/SQL and relational database validation.
• Experience with workflow schedulers such as Control-M or CA-7.
• Strong Unix/Linux scripting experience.
• Experience integrating QA into CI/CD pipelines using Jenkins and Git.
• Familiarity with DevOps practices and automated pipeline testing.
• Experience with container platforms (Docker / OpenShift).
• Strong analytical and debugging skills, especially in data-related issues.

Responsibilities

• Lead QA strategy, test planning, and execution across data and application platforms.
• Design and implement automated testing frameworks using Python and PySpark for large-scale data validation.
• Validate data pipelines in Hadoop ecosystem ensuring completeness and accuracy.
• Develop end-to-end test scenarios for batch processing, ingestion, and transformation.
• Perform database validation and reconciliation using Oracle PL/SQL and SQL techniques.
• Validate ML pipelines and models with experiment tracking via MLflow.
• Monitor and validate workflows orchestrated via Control-M/CA-7.
• Embed quality gates into CI/CD with Jenkins and Git.
• Drive test automation within DevOps pipelines and release processes.
• Perform Unix-based scripting for test automation and data checks.
• Lead defect triage and root cause analysis.
• Ensure regulatory and audit compliance in data-sensitive environments.
• Mentor QA engineers and promote best practices in test automation.

Skills

Python

PySpark

Hadoop

Hive

Impala

PL/SQL

Control-M

CA-7

Unix scripting

Jenkins

Git

Docker

OpenShift

Education

Bachelor of Computer Science

Tools

Jenkins

Git

Docker

OpenShift

Control-M

CA-7

Job description

Tata Consultancy Services seeks a Quality Assurance Lead (Contractor) to drive end-to-end testing strategy for large-scale data platforms. You will lead QA efforts across batch and streaming environments, focusing on data integrity, regulatory compliance, and system reliability.

The role requires strong Python/PySpark automation, experience with Hadoop ecosystems, and expertise in CI/CD integration with Jenkins and Git.