ETL Tester PySpark | Databricks Data Validation

Valiant Technologies LLC; Boston (MA); On-site; USD 90,000 - 120,000; Full time

Job summary

Valiant Technologies LLC in Boston is seeking an ETL Tester with 3–5 years of experience to join our data testing and automation initiatives. You will work with PySpark, SQL, Databricks, and API testing to validate data across systems.

You will develop basic automation scripts, execute test cases, participate in Agile ceremonies, and collaborate with engineers, analysts, and QA teams to ensure data quality and reliability.

Qualifications

• 3–5 years of experience in ETL Testing, Data Testing, or Data Automation.
• Working knowledge of PySpark and Apache Spark.
• Basic to intermediate experience with Databricks and Spark SQL.
• Good understanding of SQL queries, joins, aggregations, and analytical functions.
• Experience in data validation, data reconciliation, and ETL testing.
• Exposure to API Testing and API Automation.
• Understanding of Entity Resolution and data matching concepts.
• Knowledge of Software Testing Life Cycle (STLC).
• Experience creating and executing test cases and test scripts.
• Familiarity with JIRA, Xray, and Confluence.
• Understanding of Agile/Scrum methodologies.

Responsibilities

• Perform ETL testing and validate data across different systems.
• Execute data validation and reconciliation using SQL queries.
• Develop and maintain basic automation scripts using PySpark.
• Test data transformation and processing workflows in Databricks.
• Perform API testing and validate API responses.
• Write and execute functional, integration, and regression test cases.
• Identify, document, and track defects using JIRA.
• Validate data accuracy, completeness, and consistency.
• Support test planning, test execution, and QA documentation.
• Collaborate with developers, business analysts, and QA teams.
• Participate in Agile ceremonies and daily project meetings.
• Assist with Business User Acceptance Testing (BUA/UAT).

Skills

ETL Testing

Data Testing

Data Automation

PySpark

SQL

Databricks

API Testing

Entity Resolution

STLC

JIRA

Xray

Confluence

Agile/Scrum

Tools

Databricks

JIRA

Xray

Confluence

Job description

Valiant Technologies LLC in Boston is seeking an ETL Tester with 3–5 years of experience to join our data testing and automation initiatives. You will work with PySpark, SQL, Databricks, and API testing to validate data across systems.

You will develop basic automation scripts, execute test cases, participate in Agile ceremonies, and collaborate with engineers, analysts, and QA teams to ensure data quality and reliability.