Data Tester / Data Integration Test Engineer

HTD Talent; Dallas (TX); On-site; USD 90,000 - 120,000; Full time

Job summary

HTD Talent is seeking experienced Data Testers to validate complex data pipelines across Databricks, Kafka, and Cosmos DB. You will ensure data accuracy in near-real-time architectures and coordinate testing activities with data engineers, developers, and business teams.

Responsibilities include end-to-end testing of data flows, API and SQL validation, and defect management through resolution. Strong SQL, testing automation, and analytical skills are essential.

Qualifications

• 3+ years of data testing or data integration testing experience.
• Hands-on with Databricks for data pipeline testing and validation.
• Hands-on with Apache Kafka including event consumption and streaming validation.
• Hands-on with Azure Cosmos DB including data validation and integration testing.
• Experience testing near-real-time data processing and event-driven architectures.
• Experience validating end-to-end data flows involving Kafka, Azure Functions, and Cosmos DB.
• Strong SQL skills for source-to-target validation and data reconciliation.
• Experience with API testing and automated test scripts.
• Defect management, troubleshooting, and production deployment validation.
• Strong communication and analytical skills.

Responsibilities

• Design and execute test plans, cases, and automated scripts based on requirements.
• Perform end-to-end data integration testing across Databricks, Kafka, Azure Functions, Cosmos DB.
• Validate Kafka event consumption, processing, and data loading into Cosmos DB.
• Test near-real-time data pipelines for accuracy and completeness.
• Perform API, functional, and integration testing.
• Write and execute SQL queries to validate data transformations.
• Prepare test data, execute tests, and document results.
• Identify, log, and track defects through resolution.
• Support defect triage and post-release validation.
• Collaborate with data engineers, developers, and business teams to resolve issues.

Skills

Data testing

Databricks

Apache Kafka

Azure Cosmos DB

End-to-end data validation

SQL

API testing

Defect management

Communication & analysis

Tools

Azure Event Hubs

ADLS Gen2

PySpark

Job description

We are seeking experienced Data Testers with strong hands-on experience in Databricks, Kafka, Cosmos DB, and near-real-time data processing. This role will focus on testing complex data pipelines, validating integrations, and ensuring data accuracy across cloud-based and event-driven architectures.

Candidates must have experience testing end-to-end data flows, particularly Kafka event consumption, processing through Azure Function Apps, and data loading into Cosmos DB.

Key Responsibilities

• Design and execute test plans, test cases, and automated test scripts based on business and technical requirements.
• Perform end-to-end data integration testing across Databricks, Kafka, Azure Function Apps, and Cosmos DB.
• Validate Kafka event consumption, processing, and successful data loading into Cosmos DB.
• Test near-real-time data pipelines to ensure accuracy, completeness, and reliable source-to-target data flow.
• Perform API testing, functional testing, and integration testing.
• Write and execute SQL queries to validate data transformations, business rules, and data quality.
• Prepare test data, execute test cases, and document results.
• Identify, log, troubleshoot, and track defects through resolution.
• Support defect triage, production deployment verification, and post-release validation.
• Collaborate with data engineers, developers, and business teams to resolve integration and data quality issues.

Required Qualifications

• 3+ years of experience in data testing, ETL/ELT testing, or data integration testing.
• Hands-on experience with Databricks, including data pipeline testing and validation.
• Hands-on experience with Apache Kafka, including event consumption and streaming data validation.
• Hands-on experience with Azure Cosmos DB, including data validation and integration testing.
• Experience testing near-real-time data processing and event-driven architectures.
• Experience validating end-to-end data flows involving Kafka, Azure Function Apps, and Cosmos DB.
• Strong SQL skills, including source-to-target validation and data reconciliation.
• Experience with API testing and automated test scripts.
• Experience with defect management, troubleshooting, and production deployment validation.
• Strong communication and analytical skills.

Preferred Qualifications

• Experience with Azure Event Hubs and Azure Data Lake Storage (ADLS Gen2).
• Experience with PySpark and distributed data processing.
• Experience building automated data testing frameworks.
• Familiarity with Azure cloud architecture and data integration services.