Principal Data Architect - AI-Driven Data Fabric

Medidata Solutions; Greater London; Hybrid; GBP 110,000 - 150,000; Full time

Benefits offered by this job

Medical insurance

Dental insurance

Life insurance

Disability insurance

Pension

25+ holidays

Job summary

Medidata in London is seeking a Principal Data Engineer / Architect to lead the strategic vision and hands-on execution of an object-centric data fabric, unifying multi-stream clinical data such as EDC, telemetry, and real-world datasets.

You will build a modern Data Lake using Apache Iceberg, Snowflake, and Kafka, develop backend services in Java or Scala on AWS, and apply AI-assisted tooling for schema mapping, testing, and validation, while ensuring GxP, HIPAA and GDPR compliance.

Qualifications

• Education & Experience: Bachelor's or Master's degree in Computer Science, Data Science, Software Engineering, or equivalent practical experience.
• Data Warehousing & Data Lake Mastery: Snowflake, Iceberg, data lake structures.
• Software Engineering & Streaming: Production backend services in Java or Scala; Kafka; AWS.
• AI-Augmented Engineering Proficiency: AI tools integrated into daily workflows for SQL and code tasks.
• Engineering Rigor & Methodologies: TDD/BDD; CI/CD; Git.
• Domain & Regulatory Awareness: Knowledge of clinical trial data workflows and compliance standards (GxP, HIPAA, GDPR).

Responsibilities

• Architect data fabric and evolve enterprise data lake with Iceberg as core storage.
• Develop multi-stream ingestion pipelines and AI-driven schema mapping artifacts.
• Build real-time pipelines with Kafka, AWS, Snowflake; write enterprise services in Java/Scala.
• Promote technical direction and optimize Snowflake/Iceberg SQL workloads.
• Integrate AI inference into data pipelines for anomaly detection and mapping safety signals.
• Lead TDD/BDD initiatives and generate validation artifacts and audit documentation.

Skills

SQL proficiency

Java

Scala

Kafka

Snowflake

Iceberg

AWS

AI tools

CI/CD

TDD/BDD

Regulatory awareness

Education

Bachelor's or Master's degree in Computer Science, Data Science, Software Engineering, or equivalent practical experience

Tools

SQL

Snowflake

Apache Iceberg

Kafka

Java

Scala

AWS

Job description

Medidata in London is seeking a Principal Data Engineer / Architect to lead the strategic vision and hands-on execution of an object-centric data fabric, unifying multi-stream clinical data such as EDC, telemetry, and real-world datasets.

You will build a modern Data Lake using Apache Iceberg, Snowflake, and Kafka, develop backend services in Java or Scala on AWS, and apply AI-assisted tooling for schema mapping, testing, and validation, while ensuring GxP, HIPAA and GDPR compliance.