Data Engineer - Spark & Scala | Hybrid Glasgow

GIOS Technology Limited; Glasgow; Hybrid; GBP 60,000 - 90,000; Full time

Job summary

GIOS Technology Limited in Glasgow, UK, is seeking a Data Engineer specialized in Spark and Scala to design, build and maintain large-scale data pipelines. The role operates in a hybrid model (2–3 days onsite) and focuses on data processing, validation and orchestration using Airflow.

The ideal candidate will be proficient in Python, SQL and PostgreSQL, with production support experience and a strong focus on data quality, monitoring and resilient pipelines.

Qualifications

• Hands-on Spark and Scala for large-scale data processing.
• Experience building data pipelines and orchestration using Airflow.
• Proficiency in Python, SQL, and PostgreSQL for data processing and validation.
• Experience with production support and debugging failed workflows.
• Familiarity with GitLab, CI/CD, Docker and release practices.
• Understanding of data quality, monitoring and operational resilience.
• Nice to have: Starburst/Trino, AWS S3, Kubernetes or OpenShift.

Responsibilities

• Develop and maintain data pipelines and orchestration workflows using Airflow.
• Transform and validate large-scale datasets with Spark and Scala.
• Process data with Python, SQL, and PostgreSQL; troubleshoot issues.
• Investigate production incidents, data mismatches and missing files.
• Collaborate with teams using GitLab, CI/CD, Docker and release practices.
• Ensure data quality, monitoring, logging, auditability and resilience.
• Familiarity with Starburst/Trino, AWS S3, Kubernetes or OpenShift is a plus.

Skills

Spark

Scala

Python

Airflow

SQL

PostgreSQL

Data Engineering

Production Support

Tools

GitLab

CI/CD

Docker

Kubernetes

OpenShift

AWS S3

Starburst/Trino

Job description

GIOS Technology Limited in Glasgow, UK, is seeking a Data Engineer specialized in Spark and Scala to design, build and maintain large-scale data pipelines. The role operates in a hybrid model (2–3 days onsite) and focuses on data processing, validation and orchestration using Airflow.

The ideal candidate will be proficient in Python, SQL and PostgreSQL, with production support experience and a strong focus on data quality, monitoring and resilient pipelines.