Data QA Engineer: Databricks, PySpark & Data Pipelines

Harnham; Greater London; On-site; GBP 50,000 - 70,000; Full time

Job summary

Harnham is seeking a Data QA Engineer to shape quality standards across a modern data platform within a large-scale transformation programme. You will influence testing strategy and work closely with data engineers and analytics engineers in a highly collaborative environment.

The role involves testing data platforms, designing tests, managing defects, and driving improvements across testing processes and CI/CD pipelines.

Qualifications

• Strong commercial experience within Quality Assurance, Software Testing or Data Testing environments.
• Proven experience testing data platforms, data pipelines and complex technical solutions.
• Hands-on experience with Databricks.
• Strong knowledge of PySpark.
• Experience using Python within testing or data-focused environments.
• Ability to work independently and take ownership of testing activities.
• Strong stakeholder management and communication skills.
• Experience working within Agile delivery environments.
• Exposure to CI/CD pipelines and automated deployment processes.
• Experience with test automation frameworks.
• Understanding of data quality principles and validation techniques.
• Experience working within consulting or project-based delivery environments.

Responsibilities

• Test data platforms, pipelines and associated data products
• Design and execute functional, integration and regression testing activities
• Develop testing strategies and supporting quality assurance best practices
• Work closely with Data Engineers and Analytics Engineers throughout delivery
• Identify, log, track and manage defects through to resolution
• Drive continuous improvement across testing processes, frameworks and standards
• Contribute to quality discussions and helping define effective testing approaches
• Work within a Databricks environment, collaborating on both testing and development activities
• Partner with engineering teams working extensively with PySpark
• Communicate effectively with technical and business stakeholders across the organisation

Skills

Quality Assurance

Data Testing

Agile Delivery

Stakeholder communication

Independent ownership

Tools

Databricks

PySpark

Python

CI/CD pipelines

Test automation frameworks

Job description

Harnham is seeking a Data QA Engineer to shape quality standards across a modern data platform within a large-scale transformation programme. You will influence testing strategy and work closely with data engineers and analytics engineers in a highly collaborative environment.

The role involves testing data platforms, designing tests, managing defects, and driving improvements across testing processes and CI/CD pipelines.