Data Pipeline Architect — Core Data Engineer

Optima Partners; City of Edinburgh; On-site; GBP 60,000 - 85,000; Full time

Job summary

Optima Partners is seeking a Core Data Engineer to design, build, and optimise data pipelines, architectures, and workflows for a diverse client base in Edinburgh, UK. You will work in cross-functional teams and contribute to scalable data solutions while adopting best practices and emerging trends in data engineering.

You will collaborate with data scientists, analysts, and stakeholders, delivering robust ETL/ELT processes, data integration, and documentation.

Qualifications

• 2+ years of data engineering experience.
• Proven ability to deliver data solutions in consulting or client-facing environments is a plus.
• Experience with Agile or Scrum methodologies is beneficial.

Responsibilities

• Data Pipeline Development: Design and implement robust, scalable data pipelines to collect, transform, and integrate data from various sources.

Skills

Python

Java

Scala

Databricks

Apache Spark

Hadoop

AWS

Azure

GCP

PostgreSQL

SQL Server

Airflow

NiFi

Talend

Snowflake

Redshift

BigQuery

Git

Docker

CI/CD

2+ years data engineering

Consulting experience

Problem-solving

Communication

Teamwork

Data pipelines

Tools

Databricks

Job description

Optima Partners is seeking a Core Data Engineer to design, build, and optimise data pipelines, architectures, and workflows for a diverse client base in Edinburgh, UK. You will work in cross-functional teams and contribute to scalable data solutions while adopting best practices and emerging trends in data engineering.

You will collaborate with data scientists, analysts, and stakeholders, delivering robust ETL/ELT processes, data integration, and documentation.