Data Engineer: Real-Time ETL, ML & Data Services (Hybrid)

Smarkets; Greater London; Hybrid; GBP 70,000 - 110,000; Full time

Benefits offered by this job

Stock Options

Pension Scheme

Health Insurance

Enhanced Parental Leave

Breakfast & Lunch

Cycle-to-Work

Education Budget

Holiday: 25 days

Job summary

Smarkets is seeking a Data Engineer to join its Data Team in London. You will develop and maintain real-time and batch ETL pipelines, help keep our data lake and warehouses healthy, and build Flask services with Postgres databases to enable data access across teams.

You will explore data, build ML models, and contribute to a recommender service that informs user engagement. A hybrid work model is offered with 3 days in the office and 2 from home.

Qualifications

• 3+ years of experience in data engineering or equivalent role.
• Experience building ETL pipelines with Python frameworks (Luigi/Airflow).
• Experience developing Python-based REST APIs/services and integrating with Postgres.
• Familiar with Pandas, NumPy and scikit-learn.
• Some experience with training and deploying ML models.
• Ability to write elegant, well-tested code and work in a team.
• Bachelor’s degree in Computer Science or equivalent field, or relevant experience.

Responsibilities

• Developing and maintaining Data ETL pipelines (some real-time).
• Ensuring the data lake and warehouses (Redshift/BigQuery) are healthy.
• Developing Flask services and Postgres databases for data access.
• Assisting teams with automated reporting pipelines.
• Exploring data and training/deploying ML models, incl. recommender service improvements.

Skills

3+ years experience

ETL pipelines with Python (Luigi/Airflo

Python REST APIs

Pandas/NumPy/Scikit-learn

ML model training/deploying

Clean, well-tested code

Team player

Education

Bachelor in Computer Science or equivalent

Tools

Luigi

Airflow

Flask

Postgres

Kubernetes

Docker

Python

Job description

Smarkets is seeking a Data Engineer to join its Data Team in London. You will develop and maintain real-time and batch ETL pipelines, help keep our data lake and warehouses healthy, and build Flask services with Postgres databases to enable data access across teams.

You will explore data, build ML models, and contribute to a recommender service that informs user engagement. A hybrid work model is offered with 3 days in the office and 2 from home.