Data Engineer – Real‑Time Pipelines & AI (Hybrid)

Opus Recruitment Solutions; West of England; Hybrid; GBP 60,000 - 71,000; Full time

Benefits offered by this job

Bonus scheme (10%)

Health insurance

Pension scheme

33 days holiday

Job summary

Opus Recruitment Solutions in Bristol is seeking a Data Engineer to join a hybrid team. The role focuses on building scalable data pipelines using Python, SQL and PySpark, with Databricks, Kafka and AWS in the stack.

You will work on real-time platforms processing billions of sensor readings monthly, collaborating with Data Scientists and Software Engineers to shape the data architecture and drive AI/ML initiatives.

Qualifications

• Strong commercial experience as a Data Engineer.
• Advanced Python and SQL skills.
• Strong PySpark experience.
• Experience building and maintaining scalable data pipelines.
• Understanding of data modelling and database design principles.
• Strong analytical and problem-solving capabilities.
• Excellent communication and stakeholder engagement skills.
• Degree in Computer Science, Mathematics, Statistics or a related discipline would be advantageous.

Responsibilities

• Build and optimise large-scale data pipelines.
• Work with billions of data points generated from IoT devices.
• Develop solutions using Python, SQL and PySpark.
• Gain exposure to Databricks, Kafka and streaming technologies.
• Contribute to Machine Learning and AI initiatives.
• Work alongside Data Scientists and Software Engineers.
• Influence architecture and engineering best practices.
• Play a key role in exciting greenfield projects.

Skills

Python

SQL

PySpark

Data pipelines

Stakeholder engagement

Education

Bachelor's degree in Computer Science, Mathematics, Statistics or related

Tools

Databricks

Kafka

AWS

Job description

Opus Recruitment Solutions in Bristol is seeking a Data Engineer to join a hybrid team. The role focuses on building scalable data pipelines using Python, SQL and PySpark, with Databricks, Kafka and AWS in the stack.

You will work on real-time platforms processing billions of sensor readings monthly, collaborating with Data Scientists and Software Engineers to shape the data architecture and drive AI/ML initiatives.