Data Engineer: Real-Time Pipelines & ML Services

Smarkets; Greater London; On-site; GBP 50,000 - 70,000; Full time

Benefits offered by this job

Stock Options

Pension Scheme with matched contributions

Health Insurance

Enhanced Parental Leave

Fresh meals and snacks daily

Cycle-to-Work Scheme

Annual education budget

25 days paid leave

Flexible Working options

Regular team events and socials

Job summary

A leading prediction market technology firm in Greater London is seeking an experienced Data Engineer. In this role, you will develop and maintain ETL pipelines, manage data lakes, and assist with automated reporting. You should have over 3 years of experience in data engineering, knowledge of Python frameworks, and a bachelor's degree in a relevant field. Join a diverse team that values innovation and collaboration, and be part of a supportive work culture with competitive benefits.

Qualifications

• 3+ years of experience in data engineering or related field.
• Experience developing ETL pipelines with Python.
• Understanding of Python-based REST APIs.

Responsibilities

• Develop and maintain ETL pipelines for business insights.
• Manage data lakes and warehouses to ensure data health.
• Assist teams with automated reporting and data access.

Skills

ETL pipeline development

Python programming

Team collaboration

Data analysis

Machine learning

Education

Bachelor's degree in Computer Science or Math

Tools

Postgres

Python frameworks (e.g., Luigi, Airflow)

Flask

AWS Redshift

Google BigQuery

Job description

A leading prediction market technology firm in Greater London is seeking an experienced Data Engineer. In this role, you will develop and maintain ETL pipelines, manage data lakes, and assist with automated reporting. You should have over 3 years of experience in data engineering, knowledge of Python frameworks, and a bachelor's degree in a relevant field. Join a diverse team that values innovation and collaboration, and be part of a supportive work culture with competitive benefits.