Data-Driven Test Automation Engineer for ETL, Spark & CI/CD

Trinity Consulting Services; Singapore; On-site; SGD 70,000 - 110,000; Full time

Job summary

Trinity Consulting Services is seeking an experienced Software Test Engineer to join our data-centric QA team. You will validate ETL pipelines, perform data validation and reconciliation, and ensure data quality across large datasets using SQL, Spark SQL, and automated testing tools.

The role requires hands-on experience with Java/Spark-based processing, automation frameworks, and CI/CD pipelines. You will diagnose performance issues and contribute to test reporting and data-driven testing

Qualifications

• Minimum 4–6 years’ relevant experience in software testing.
• Strong understanding and working knowledge of Agile/Scrum.
• Extensive ETL testing, data validation, data reconciliation, and source-to-target testing experience.
• Hands-on MS SQL Server and advanced T‑SQL experience.
• Experience testing Java/Spark-based data processing applications.
• Hands-on automation using Java, JUnit/TestNG, Maven/Gradle, and data-driven test frameworks.
• Ability to validate large datasets using SQL, Spark SQL, and automated comparison utilities.
• Experience with batch processing, incremental loads, data quality checks, and regression testing.
• Familiarity with CI/CD tools like Jenkins, GitLab CI, Azure DevOps, or GitHub Actions.
• Experience with Git, scripting (PowerShell/Bash/Python), and automated test reporting.
• Ability to analyze ETL logs, Spark job logs, and SQL Server performance issues.
• Knowledge of file formats (CSV, JSON, XML, Parquet, Avro, fixed-width).
• Good test data management, data masking, and production-like test data setup.

Skills

Software testing

Agile/Scrum

ETL testing

Data validation

Data reconciliation

Source-to-target testing

SQL Server

T-SQL

Java

Spark

Automation testing

JUnit/TestNG

Maven/Gradle

Jenkins

GitLab CI

Azure DevOps

GitHub Actions

Git

Scripting (PowerShell/Bash/Python)

Log analysis

CSV/JSON/XML/Parquet/Avro

Tools

MS SQL Server

T‑SQL

JUnit/TestNG

Maven

Gradle

Jenkins

GitLab CI

Azure DevOps

GitHub Actions

PowerShell

Bash

Python

Job description

Trinity Consulting Services is seeking an experienced Software Test Engineer to join our data-centric QA team. You will validate ETL pipelines, perform data validation and reconciliation, and ensure data quality across large datasets using SQL, Spark SQL, and automated testing tools.

The role requires hands-on experience with Java/Spark-based processing, automation frameworks, and CI/CD pipelines. You will diagnose performance issues and contribute to test reporting and data-driven testing