ETL QA

Kumaran Systems; Toronto; On-site; CAD 90,000 - 120,000; Full time

Job summary

Kumaran Systems seeks an ETL QA Engineer with strong Databricks and PySpark experience to validate large-scale data pipelines, ensure data quality, and apply AI-assisted testing methodologies. You will work with CI/CD utilities, GitHub Copilot, and AI agents to improve coverage and efficiency, while contributing to AI-driven knowledge bases.

The role emphasizes QA best practices, data modeling understanding, and collaboration across AI, data engineering, and operations teams in a fast-paced,

Qualifications

• Proven experience validating large-scale data pipelines and ensuring data quality.
• Hands-on exposure to AI-assisted development and prompt engineering.
• Experience building AI-driven knowledge bases or documentation systems.
• Experience with automated testing frameworks (pytest, unittest).

Responsibilities

• Validate large-scale data pipelines and ensure data quality across datasets.
• Leverage tools like GitHub Copilot and GitHub Actions/Runners to improve testing efficiency and coverage.
• Contribute to building and validating AI-powered knowledge bases.
• Collaborate with data engineers, QA, and AI teams to implement robust QA processes.

Skills

Databricks

PySpark

ETL/ELT concepts

Data warehousing modeling

Data validation & quality checks

GitHub Copilot & Actions

Tools

Delta Lake

GitHub

GitHub Copilot

CI/CD pipelines

Job description

About Us

Kumaran Systems: Engineering Future-Ready Transformations Since 1992, Kumaran Systems has been helping enterprises bridge the past and the future, reimagining legacy systems and embracing AI-led possibilities. With over 34 years of delivery excellence, we specialise in re-engineering core applications, driving cloud transformation, and building automation-first, GenAI-enabled ecosystems across the Banking and Financial Services, Insurance, Telecom, and Automotive sectors. We help organisations not just upgrade their systems but reimagine what’s possible.

About Us

Kumaran Systems: Engineering Future-Ready Transformations Since 1992, Kumaran Systems has been helping enterprises bridge the past and the future, reimagining legacy systems and embracing AI-led possibilities. With over 34 years of delivery excellence, we specialise in re-engineering core applications, driving cloud transformation, and building automation-first, GenAI-enabled ecosystems across the Banking and Financial Services, Insurance, Telecom, and Automotive sectors. We help organisations not just upgrade their systems but reimagine what’s possible.

Why Kumaran?

We blend engineering discipline with AI innovation, helping clients modernise with confidence, automate with clarity, and scale with purpose. Our global delivery model ensures agility, responsiveness, and seamless collaboration, with clients always at the heart of every engagement. At Kumaran, we don’t just solve problems, we engineer future-ready transformations.

Role Overview

We are looking for an ETL QA Engineer with strong experience in Databricks and PySpark, combined with hands-on exposure to AI-assisted development and prompt engineering. The ideal candidate will be responsible for validating large-scale data pipelines, ensuring data quality, and leveraging tools like GitHub Copilot, GitHub Actions/Runners, and AI agents to improve testing efficiency and coverage. You will also contribute to building and validating AI-powered knowledge bases.

Required Skills & Experience

• Core Technical Skills
  • Strong hands-on experience in Databricks (notebooks, jobs, clusters, Delta Lake, SQL).
  • Expert-level PySpark skills, including transformations, actions, window functions, and optimization techniques.
  • Solid understanding of ETL/ELT concepts, data warehousing, and data modeling (star/snowflake schemas, dimension/fact tables).
  • Proven experience in data validation, data quality checks, and reconciliation techniques.
  • Practical experience with GitHub, GitHub Copilot, and GitHub Actions/Runners for CI/CD pipelines.
• AI & Prompt Engineering
  • Experience crafting effective prompts for large language models (LLMs) for coding, testing, and documentation.
  • Exposure to agent-based architectures or tools (e.g., workflow/agent frameworks that orchestrate multi-step AI tasks).
  • Familiarity with AI-based tools for code analysis, test generation, or knowledge management.
• Knowledge Base & Documentation
  • Experience building AI-driven knowledge bases or documentation systems (e.g., using vector search, embeddings, or LLM-based retrieval).
  • Strong documentation skills, with the ability to translate complex data/QA concepts into clear knowledge articles.
• Testing & QA Practices
  • Strong understanding of QA methodologies: test planning, test design, defect management, and regression testing.
  • Experience with automated testing frameworks (Python-based testing libraries such as pytest , unittest , or similar).
  • Familiarity with data quality tools/practices (e.g., validation rules, thresholds, anomaly detection).

Key Attributes

• Strong analytical and problem-solving skills with a detail-oriented mindset.
• Passion for data quality, automation, and continuous improvement.
• Ability to work in an agile, fast-paced environment and collaborate across multiple teams like Business and operations team and AD team.
• Curiosity and openness to adopting new AI tools and practices for QA.