Principal Data Consultant - AWS / Databricks

The Scale Factory Ltd; Greater London; Hybrid; GBP 110,000 - 140,000; Full time

Benefits offered by this job

25 Days of Annual Leave

Pension plan

Health benefits

Gym subsidies

Commuter benefits

Free Annual Rail Cards

EV salary sacrifice

Job summary

The Scale Factory Ltd. is seeking a seasoned Data Architect/Engineer to lead architecture work on high-impact data engagements. You will define visions with senior clients, defend designs with CTOs, and guide hands-on coding when needed.

You’ll shape reusable reference architectures and delivery standards for the practice. You will own target data architecture, governance, cost models, and security from inception to delivery, while mentoring engineers and advancing the practice within a hybrid

Qualifications

• 10+ years in data engineering and architecture on multi-team programmes.
• Experience delivering modern data platforms end-to-end on AWS/Databricks.
• Production-grade code in Python/SQL; IaC (Terraform).

Responsibilities

• Lead architecture work on client engagements from discovery to production.
• Run discovery and target-architecture workshops with CTOs and data/engineering leads.
• Translate commercial goals into architecture and a funded roadmap.
• Act as senior technical voice for trade-offs, cost and risk.

Skills

Data architecture

Data engineering

Python

SQL

Spark

dbt

Terraform

Client-facing

Education

Bachelor's in CS/SE

Tools

AWS

Databricks

Kafka

Kinesis

Job description

About Scale Factory

We are an AI enablement consultancy. Delivering quality-led AI-first foundations, embedding capability and excellence into the foundational fabric of the companies we work with. We give leaders the confidence to move faster, take smarter risks and pursue transformation.

Formerly Ten10, we have deep roots in Quality which means we don’t just build for the future, we ensure it actually works. We combine this engineering discipline with cutting-edge AI expertise to deliver robust, scalable foundations that turn ambitious transformation into reliable reality.

Role purpose

You’ll be the architect on our highest-impact data engagements. You will define the vision with senior clients, defend your design in front of CTOs and engineering teams, and stay grounded in the codebase to prove it works. When the project demands it, you’ll step up as Tech Lead. Off-project, you’ll help shape our practice: taking real-world learnings and building the reference architectures, repeatable propositions, and delivery standards that set the benchmark for everything we do next.

Key responsibilities:

Client engagement leadership

• Lead the architecture work on client engagements from discovery through to production
• Run discovery and target-architecture workshops with client CTOs, heads of data and engineering leads
• Translate commercial goals into an architecture and a sequenced roadmap the client can fund
• Be the senior technical voice the client trusts when trade-offs, cost or risk are contested

Architecture ownership

• Set and own the target data architecture: ingestion, storage, transformation, serving, governance
• Make and document the significant decisions (lakehouse vs warehouse, batch vs streaming, build vs buy) with the trade-offs written down
• Define non-functional requirements, cost models and FinOps guardrails alongside the functional design
• Build in security, data protection, lineage and quality from the start, not as a later workstream

Hands-on technical leadership

• Take the tech lead role on engagements when the shape of the team requires it
• Write and review code where it matters: reference implementations, spikes, thorny pipelines, infrastructure-as-code
• Set engineering standards for the practice — testing, CI/CD, environments, code review, observability
• Coach and unblock engineers; raise the standard of the people around you

Practice building and pre-sales

• Spot and qualify follow-on work inside live accounts
• Build reusable assets: reference architectures, accelerators, delivery playbooks
• Help define the hiring bar and interview the next data engineers and architects
• Work with the existing cloud infrastructure and testing practices so data engagements pull them through, and vice versa

Essential experience

• 10+ years in data engineering and architecture, with at least 3 as the lead architect on complex, multi-team programmes
• Designed and delivered modern data platforms end to end on AWS and/or Databricks
• Still writes production-grade code: strong Python and SQL, Spark, dbt or equivalent, infrastructure-as-code (Terraform preferred)
• Track record in a consultancy, systems integrator or other client-facing setting — comfortable being accountable to a paying client, not only an internal stakeholder
• Data modelling depth across dimensional, Data Vault and lakehouse/medallion approaches, with a view on when each is most appropriate
• Data governance in practice: catalogue, lineage, access control, quality, and Regulatory obligations
• Must have a conceptual appreciation of AI systems and a willingness to learn
• Owned platform cost: sizing, forecasting and bringing a runaway bill back down
• Estimating and scoping work, and shaping proposals that were won
• SC clearance, or eligibility for it

Desirable

• Databricks and/or AWS certification at professional level, and experience supporting a firm's partner status (competency badges, certified-headcount targets, co-sell with partner account teams)
• Streaming and event-driven architecture in production (Kafka/MSK, Kinesis, Structured Streaming)
• Data products, mesh or federated ownership models applied on a real programme, not just in theory
• ML and AI platform work: feature stores, MLflow, model serving, and the data foundations behind RAG or agentic use cases
• Regulated-sector exposure — financial services, healthcare, public sector — and the assurance that comes with it

How you work

• Opinionated but not dogmatic: recommends the simplest architecture that meets the requirement, and says so when a client's preferred tool is the wrong one
• Comfortable being the only data person in the room and building the case from scratch
• Writes clearly — decision records, options papers and diagrams a client executive can act on
• Commercially aware: understands utilisation, margin and how a scope conversation protects both sides
• Generous with knowledge; wants to grow a team rather than be its single point of failure
• Happy with the ambiguity of an early-stage practice inside an established business

What’s in It for You?

At Scale Factory, we believe in recognising and rewarding great work. Here’s what you can expect:

• 25 Days of Annual Leave: Plus 1 extra day every year for the first three years, with a holiday buying scheme.
• Pension Plan: Employer-matched contributions up to 5%.
• Health Benefits: Comprehensive coverage for medical, dental, optical, and alternative therapies.
• Fitness Support: Subsidised gym memberships and a bike-to-work scheme.
• Commuter Benefits: Season ticket loans to make travel easier.
• Free Annual Rail Cards
• Electric vehicle salary sacrifice scheme
• Flexible Work Model: Hybrid working that fits your lifestyle.
• Social Events: From annual kick-offs and Christmas parties to team socials and sporting events, we love celebrating success together.
• Training & Development: Tailored learning opportunities and full support for certifications to keep your skills sharp.

Let’s Build the Future Together