Data Engineer

LA International; West Midlands; Hybrid; GBP 60,000 - 95,000; Full time

Benefits offered by this job

Private healthcare

Life assurance

Critical illness cover

Enhanced pension contributions

Personal development and training

Job summary

LA International is seeking a Data & AI Engineer to join a business undergoing an enterprise data transformation in the UK. You will work on robust data platforms, pipelines, and AI prototypes, leveraging Databricks capabilities and Delta Lake, while advising clients on what is possible with their data.

You'll collaborate with customers to understand business challenges, develop proofs of concept, and apply Generative AI concepts to real problems, with a focus on data quality and governance.

Qualifications

• Commercial experience with Databricks.
• Proficient in Python, SQL and PySpark.
• Experience with Spark and lakehouse technologies.
• Understanding of data quality, governance and data architecture.
• Knowledge of Generative AI and its practical applications.
• Strong communication skills and confidence working with stakeholders.
• Unity Catalog knowledge.
• Experience building AI-enabled business applications.

Responsibilities

• Build and optimise data pipelines using Python, SQL and PySpark.
• Develop scalable solutions in Databricks and Delta Lake.
• Create the data foundations needed for analytics and AI initiatives.
• Explore capabilities such as Databricks Apps and Genie.
• Build proof of concepts and prototypes around AI use cases.
• Work directly with customers to understand business challenges.
• Assess the effectiveness of new solutions and make recommendations.
• Share best practice and support other engineers within the team.

Skills

Databricks

Python

SQL

PySpark

Spark

Delta Lake

Generative AI

Data governance

Data architecture

Communication

Stakeholder management

Tools

Unity Catalog

Databricks Apps

Genie

Power Platform

Power Automate

Job description

I'm working with a business going through an enterprise data transformation.

They're looking for a Data & AI Engineer with strong Databricks experience to join a team that's helping make better use of their data, while exploring how emerging AI technologies can solve real business problems.

This isn't a pure data engineering role.

Alongside building robust data platforms and pipelines, you'll have the chance to get hands-on with the latest Databricks capabilities, developing prototypes, testing AI-driven solutions and helping clients understand what's possible with their data.

What you'll be doing

• Building and optimising data pipelines using Python, SQL and PySpark
• Developing scalable solutions in Databricks and Delta Lake
• Creating the data foundations needed for analytics and AI initiatives
• Exploring capabilities such as Databricks Apps and Genie
• Building proof of concepts and prototypes around AI use cases
• Working directly with customers to understand business challenges
• Assessing the effectiveness of new solutions and making recommendations
• Sharing best practice and supporting other engineers within the team

Required Experience

• Commercial experience with Databricks
• Python, SQL and PySpark skills
• Experience working with Spark and lakehouse technologies
• Understanding of data quality, governance and data architecture
• Knowledge of Generative AI and its practical applications
• Strong communication skills and confidence working with stakeholders
• Unity Catalog knowledge
• Power Platform or Power Automate exposure
• Experience building AI-enabled business applications

What's in it for you?

• Remote working (1 day per month in the office)
• 25 days holiday + bank holidays
• Private healthcare, life assurance and critical illness cover
• Enhanced pension contributions
• Personal development and training opportunities
• The opportunity to work with some of the latest Data & AI technologies in the market

If you're a Data Engineer who enjoys working with modern cloud data platforms and wants to be involved in the AI space as it continues to evolve, I'd be keen to speak with you.