Data & Ai Engineer (Databricks)

Aiimi; Milton Keynes; Hybrid; GBP 70,000 - 110,000; Full time

Benefits offered by this job

25 days annual leave

Flexible working options – hybrid

Mental health and wellbeing support

Annual wellbeing allowance

Up to 10% of salary in employee perks

Generous pension contribution

Ongoing professional development

Job summary

Aiimi is a British technology and services company focused on turning enterprise data into trusted assets. We seek a Data & AI Engineer to help clients build trusted data products and evaluate AI-enabled solutions, centred on Databricks.

You’ll develop data foundations and test ideas using Databricks features, collaborating with clients and colleagues to turn business problems into practical data-driven solutions.

Qualifications

• Design and build data pipelines and transformations.
• Strong Python and SQL skills with PySpark experience.
• Hands-on Databricks experience and Delta Lake.
• Experience with lakehouse architectures.
• Generative AI knowledge and practical evaluation.
• Ability to build and assess prototypes for clients.
• Clear communication with non-specialists.

Responsibilities

• Design and build data pipelines and transformations on Databricks using Python, SQL and PySpark.
• Develop reliable data foundations for analytics and AI, using Spark and Delta Lake.
• Explore Databricks capabilities such as Apps and Genie for real business needs.
• Build prototypes combining data with Databricks features.
• Work with business users to identify questions and workflows.
• Evaluate prototypes with realistic data and explain results.
• Assess data quality, structure, metadata and access implications.
• Recommend next steps for promising ideas including data/engineering work.
• Apply strong engineering practices for maintainable prototypes.
• Explain findings to clients and share reusable patterns.

Skills

Data engineering

Python

SQL

PySpark

Databricks

Delta Lake

Generative AI

Technical judgement

Communication

Tools

Databricks Apps

Genie

Azure

Job description

Job Description
Aiimi is a British technology and services company on a mission to turn enterprise data from a source of chaos and risk into a trusted, strategic asset. Through our Automated Data Governance platform and expert consultants, we help our customers get business value – from national security and critical infrastructure providers to leading global brands and government agencies.
We are looking for a Data & AI Engineer to help clients build trusted data products and evaluate AI-enabled solutions for their business. The role is centred on Databricks and combines strong data engineering with hands‑on exploration of capabilities such as Databricks Apps and Genie.
You’ll build the data foundations that make useful AI experiences possible, then work with clients and colleagues to test ideas using Databricks’ evolving features. You’ll bring the engineering judgement to build well‑structured prototypes, assess how they perform with real data, and explain what it would take to develop a promising idea further.
You’ll work with client teams, data engineers and AI specialists to understand business problems and turn them into practical solutions.

Key Responsibilities

• Design and build data pipelines and transformations on Databricks using Python, SQL and PySpark.
• Develop reliable data foundations for analytics and AI, working with technologies such as Spark and Delta Lake.
• Explore Databricks capabilities, including Apps and Genie, and identify where they could address real business needs.
• Build working prototypes that combine well‑prepared data with Databricks’ AI and application features.
• Work with business users to identify useful questions, workflows and scenarios to test.
• Evaluate prototypes using realistic examples, identify limitations and explain what affects the results.
• Investigate how data quality, structure, metadata and access permissions affect the usefulness of AI features.
• Recommend practical next steps for promising ideas, including the data, engineering and platform work needed.
• Apply strong engineering practices so prototypes are clear, maintainable and easy to build on.
• Explain technical findings to clients and colleagues and share reusable patterns across Aiimi.
• Provide technical guidance and support the development of other engineers.

The Skills You'll Bring

Essential:

• Strong data engineering experience, including designing and building data pipelines and transformations.
• Strong Python and SQL skills, with practical experience using PySpark or Spark.
• Hands‑on experience with Databricks and its data engineering capabilities.
• Experience working with lakehouse technologies such as Delta Lake.
• Experience designing data solutions with attention to quality, structure, performance and access.
• Hands‑on experience with Databricks Apps, Genie or other emerging Databricks capabilities.
• Practical understanding of generative AI, including its strengths, limitations and common failure modes.
• Ability to build and assess prototypes, interpret results and communicate what you learn.
• Strong technical judgement and the ability to work independently with clients and delivery teams.
• Clear communication skills, including the ability to explain technical choices to non‑specialists.

Desirable:

• Experience building interactive applications or data experiences within Databricks.
• Familiarity with Unity Catalog and Databricks data governance.
• Experience with Azure or Microsoft data platforms.
• Familiarity with Power Platform, Power Automate or other low‑code tools.
• Experience connecting data and AI capabilities to business workflows.
• Experience with generative AI applications, retrieval or natural‑language access to data.

Aiimi Benefits

• 25 days annual leave (plus bank holidays)
• Flexible working options – hybrid
• Mental health and wellbeing support, including access to counselling
• Annual wellbeing allowance (e.g. personal training, fitness, wellness apps)
• Up to 10% of your salary in employee benefits, including critical illness cover, life insurance, and private healthcare (post-probation)
• Generous company pension contribution
• Ongoing professional development and training opportunities