Big Data Engineer (Hybrid or Remote)

Allstate Insurance Company; United Kingdom; Remote; GBP 45,000 - 70,000; Full time

Job summary

A leading company in insurance seeks a Big Data developer to enhance their analytics capabilities. The ideal candidate will have expertise in Python and Scala, with experience in building ETL pipelines. Join a high-performing team that values innovation and offers flexible working options alongside a generous benefits package.

Benefits

Generous flexible benefits package

Access to world-class learning platforms

Clear career paths and internal mobility

People-first culture with flexible working options

Qualifications

• Minimum 3 years’ experience in Python and/or Scala.
• 2 years’ experience building big data ETL pipelines.
• Proficient in AWS Cloud and data management processes.

Responsibilities

• Develop Big Data solutions enabling advanced analytics.
• Participate in creating technical solutions using Big Data techniques.
• Partner with technology teams for analytics and data science.

Skills

Python

Scala

Big Data

Test Driven Development

Data Management

Tools

AWS

GCP

Job description

At Allstate, great things happen when our people work together to protect families and their belongings from life’s uncertainties. And for more than 90 years our innovative drive has kept us a step ahead of our customers’ evolving needs. From advocating for seat belts, air bags and graduated driving laws, to being an industry leader in pricing sophistication, telematics, and, more recently, device and identity protection.

Your role in the team

This role is responsible for executing multiple tracks of work to deliver Big Data solutions enabling advanced data science and analytics. This includes working with the team on new Big Data systems for analyzing data; the coding & development of advanced analytics solutions to make/optimize business decisions and processes; integrating new tools to improve descriptive, predictive, and prescriptive analytics.
This role contributes to the structured and unstructured Big Data / Data Science tools of Arity from traditional to emerging analytics technologies and methods. The role is responsible for assisting in the selection and development of other team members

Key Responsibilities

• Participate in the development of moderately complex and occasionally complex technical solutions using Big Data techniques in data & analytics processes

• Develops innovative solutions within the team

• Participates in the development of moderately complex and occasionally complex prototypes and department applications that integrate Big Data and advanced analytics to make business decisions

• Uses new areas of Big Data technologies, (ingestion, processing, distribution) and research delivery methods that can solve business problems

• Understands the Big Data related problems and requirements to identify the correct technical approach

• Takes coaching from key team members to ensure efforts within owned tracks of work will meet their needs

• Executes moderately complex and occasionally complex functional work tracks for the team

• Partners with Arity Technology teams on Big Data efforts

• Partners closely with team members on Big Data solutions for our data science community and analytic users

• Leverages and uses Big Data best practices / lessons learned to develop technical solutions

Essential Criteria

• All applicants must demonstrate they have a legal right to work in the UK for employment at Arity. Arity is not providing sponsorship for this vacancy.

• A minimum of 3 years’ experience in Python and/or Scala

• A minimum of 2 years’ experience with building big data ETL pipelines

• A minimum of 2 years’ experience in one or more of the following Java, Kotlin, GoLang

• Experience in Test Driven Development (TDD)

• Demonstrated experience managing large data sets

• AWS Cloud knowledge

• Clear understanding of data management, process automation, system monitoring, and maintainability.

Desirable Skills

• GCP services knowledge

• Experience building Data Lakes (preferably integration with Starburst)

Supervisory Responsibilities

• This job does not have supervisory duties.

Job Posting End Date: Friday 4th July [11:59pm]

Why join us?

Allstate NI is proud to be Allstate’s European Digital Centre of Excellence—recent winners of ‘Best Use of Cloud Services’ at the Belfast Telegraph IT Awards 2024, and recognised for our community and sustainability impact at the 2024 Business in the Community Awards and Gold accreditation for Environmental Responsibility.

We’re a product-driven, cloud-first organisation delivering real outcomes through modern technology, a digital product-centric talent model, and a culture rooted in engineering excellence. Our teams work in cross-functional structures, guided by an outcome-based delivery approach that accelerates speed, agility, and value.

We offer:

• A generous, flexible benefits package including annual leave, healthcare and dental cover, pension, and lifestyle discounts

• Access to world-class learning platforms and award-winning L&D

• Clear career paths, internal mobility, and a strong focus on growth

• A people-first culture with flexible working options

Be part of a high-performing, socially responsible organisation where your work has purpose, and your growth is supported every step of the way.

Statement on Fair Employment and Equal Opportunities

Allstate NI wishes to ensure equal opportunity is given to all job applicants. This company will not discriminate on the grounds of race, gender (including gender reassignment status), sexual orientation, religious belief, political opinion, marital status, age or disability.

We are an equal opportunities employer. We welcome applications from all suitably qualified persons. However, as women are currently under-represented in our workforce, we would particularly welcome applications from women. All appointments will be made on merit.

Applicants should note Allstate NI complete AccessNI background checks on all candidates offered a position.

Posting End Date:

05 July 2025