Lead Data Architect

RJC Group; Greater London; On-site; GBP 90,000 - 130,000; Full time

Job summary

RJC Group is seeking a senior Tech Lead to own and deliver a new enterprise data and AI platform in London. You will lead internal technology leads while coordinating external vendors, directing architecture, engineering, and code, and ensuring platform reliability across trading, risk, and finance domains.

You will design a scalable lakehouse, implement real-time streaming, and advance AI-enabled data products with governance, CI/CD, and IaC practices.

Qualifications

• Extensive experience designing and building enterprise-scale data or AI platforms.
• Owner of a comparable platform build from the ground up.
• Strong hands-on knowledge of AWS and Databricks.
• Experience with Unity Catalog, Delta Lake, Iceberg and streaming workloads.
• Production experience with Kafka, Kinesis, Flink or similar streaming tech.
• Knowledge of low-latency databases and knowledge graphs.

Responsibilities

• Building the core AWS and Databricks platform.
• Designing the lakehouse architecture using Delta and/or Iceberg.
• Delivering scalable ingestion, transformation and data-product pipelines.
• Building real-time streaming and CDC capability with Kafka, Kinesis or Flink.
• Developing semantic catalogue, lineage and knowledge-graph layers.
• Supporting governed AI, agentic and retrieval-based capabilities.
• Establishing engineering standards across CI/CD, testing, observability, GitOps and IaC.
• Leading internal technical teams and directing external vendors and delivery partners.
• Owning technical design decisions, delivery sequencing and platform reliability.
• Enabling domain teams to build and release data products independently.

Skills

Technical leadership

Enterprise data platforms

Data architecture

Cloud computing

Cross-functional collaboration

Tools

AWS

Databricks

Kafka

Kinesis

Flink

Delta Lake

Iceberg

Neo4j

Stardog

Job description

TECH LEAD – ENTERPRISE DATA & AI PLATFORM - LONDON

We’re recruiting for a senior, hands-on Technology Lead to build a new enterprise data and AI platform for a major global commodities trading organisation.

The successful candidate will have personally built a comparable data/AI platform before, not just directed one, and will bring that experience to bear leading a team of internal technology leads while directing the work of multiple external vendors and delivery partners

You’ll own the technical delivery of the platform while remaining close to the architecture, engineering and code. The environment will connect data across trading, risk, finance, operations and asset systems, creating a scalable foundation for analytics, streaming and AI use cases.

The role

You’ll take technical ownership of the platform from its initial foundations through to production delivery and wider adoption.

Key responsibilities include:

• Building the core AWS and Databricks platform
• Designing the lakehouse architecture using Delta and/or Iceberg
• Delivering scalable ingestion, transformation and data-product pipelines
• Building real-time streaming and CDC capability using technologies such as Kafka, Kinesis or Flink
• Developing the semantic, catalogue, lineage and knowledge-graph layers
• Supporting governed AI, agentic and retrieval-based capabilities
• Establishing engineering standards across CI/CD, testing, observability, GitOps and infrastructure as code
• Leading internal technical teams and providing direction to external vendors and delivery partners
• Owning technical design decisions, delivery sequencing and platform reliability
• Enabling domain teams to build and release data products independently

What we’re looking for

• Extensive experience designing and building enterprise-scale data or AI platforms
• Previous ownership of a comparable platform build from the ground up
• Strong hands-on knowledge of AWS and Databricks
• Experience with Unity Catalog, Delta Lake, Iceberg and streaming workloads
• Production experience with Kafka, Kinesis, Flink or comparable streaming technologies
• Knowledge of ClickHouse or another low-latency analytical database
• Experience designing or implementing knowledge graphs using Neo4j, Stardog or similar technology
• Strong understanding of hybrid cloud and on-premise integration
• Experience establishing robust SDLC, CI/CD, testing and infrastructure-as-code practices
• A track record of leading technical teams and managing third-party delivery partners

Experience within commodities, energy trading or financial services would be highly advantageous, particularly where this includes trading, position, P&L, exposure or market-data environments.

This role would suit someone who is still a builder at heart: technically credible at architecture and code level, comfortable leading several delivery streams and able to turn a long-term platform vision into working, production-ready releases.