Software Engineer - Data Architecture

JR United Kingdom; London; On-site; GBP 60,000 - 80,000; Full time

Job summary

A scaleup Medical AI firm in London is seeking a Software Engineer specializing in Data Architecture. The role involves developing a data lakehouse, building data transformation pipelines, and working with AWS and Databricks. Ideal candidates will have strong Python skills and experience with complex data systems. This position offers the opportunity to work on innovative projects aimed at improving drug trial success rates.

Qualifications

• Strong proficiency in Python.
• Experience building complex data transformation pipelines.

Responsibilities

• Supporting the development of the company's data lakehouse.
• Building complex data transformation pipelines.

Skills

Python

Data Transformation

AWS

Tools

Databricks

Airflow

DataWeave

Job description

Software Engineer - Data Architecture, London

We are seeking an exceptional Software Engineer - Data Architecture to join a scaleup Medical AI firm dedicated to improving drug trial success rates. With significant investment, secured partnerships, and a strong valuation, the company is attracting top talent.

Responsibilities:

1. Supporting the development of the company's data lakehouse to manage incoming data, and creating tools to translate datasets into researcher-friendly formats.
2. Building complex data transformation pipelines, including constructing intricate queries, converting data, and designing multi-step pipelines.

About You:

1. Strong proficiency in Python.
2. Experience in building complex data transformation pipelines.
3. Experience working with Databricks at scale, preferably with Iceberg.
4. Experience with Airflow or Dagster.
5. Experience with AWS and open-source technologies on top of DataWeave.
6. Desirable: Exposure to medical data, especially video or image data, beyond tabular data.

Additional Information:

EU work permit required. The position is based in London, United Kingdom. Job posting date: 12.05.2025. Application deadline: 26.06.2025.

If interested, please contact Ewan at [emailprotected] for a confidential discussion.