ETL QA Engineer - Databricks, PySpark & AI-Driven Testing

Kumaran Systems; Toronto; Presencial; CAD 90.000 - 120.000; Jornada completa

Descripción de la vacante

Kumaran Systems seeks an ETL QA Engineer with strong Databricks and PySpark experience to validate large-scale data pipelines, ensure data quality, and apply AI-assisted testing methodologies. You will work with CI/CD utilities, GitHub Copilot, and AI agents to improve coverage and efficiency, while contributing to AI-driven knowledge bases.

The role emphasizes QA best practices, data modeling understanding, and collaboration across AI, data engineering, and operations teams in a fast-paced,

Formación

• Proven experience validating large-scale data pipelines and ensuring data quality.
• Hands-on exposure to AI-assisted development and prompt engineering.
• Experience building AI-driven knowledge bases or documentation systems.
• Experience with automated testing frameworks (pytest, unittest).

Responsabilidades

• Validate large-scale data pipelines and ensure data quality across datasets.
• Leverage tools like GitHub Copilot and GitHub Actions/Runners to improve testing efficiency and coverage.
• Contribute to building and validating AI-powered knowledge bases.
• Collaborate with data engineers, QA, and AI teams to implement robust QA processes.

Conocimientos

Databricks

PySpark

ETL/ELT concepts

Data warehousing modeling

Data validation & quality checks

GitHub Copilot & Actions

Herramientas

Delta Lake

GitHub

GitHub Copilot

CI/CD pipelines

Descripción del empleo

Kumaran Systems seeks an ETL QA Engineer with strong Databricks and PySpark experience to validate large-scale data pipelines, ensure data quality, and apply AI-assisted testing methodologies. You will work with CI/CD utilities, GitHub Copilot, and AI agents to improve coverage and efficiency, while contributing to AI-driven knowledge bases.

The role emphasizes QA best practices, data modeling understanding, and collaboration across AI, data engineering, and operations teams in a fast-paced,