Data Engineer

DoorFeed; Greater London; On-site; GBP 50,000 - 70,000; Full time

Job summary

A leading data solutions provider is seeking a skilled Data Engineer to join their London office. The candidate will design, build, and maintain data pipelines, process data from numerous sources, and leverage AI and machine learning to enhance data quality. Critical qualifications include proficiency in Python and SQL, along with at least 2 years of relevant experience in data engineering. This role requires the ability to work onsite 5 days a week and will involve managing PostgreSQL databases and data transformation processes.

Qualifications

• Minimum 2 years relevant professional experience in data engineering.
• Able to work onsite in central London office 5 days a week.

Responsibilities

• Maintain and improve data pipeline of live properties.
• Manage PostgreSQL databases.
• Transform data from hundreds of different sources.
• Enrich data using AI and Machine Learning.
• Build data ingestion pipelines from scratch.

Skills

Python

SQL

pandas

data engineering technologies

problem-solving

communication skills

Tools

PostgreSQL

AWS (S3, RDS, Lambda, EC2, Athena)

machine learning frameworks

data orchestration tools (Airflow, Dagster)

Job description

The Opportunity:

We are seeking a skilled data engineer with at least 2 years experience to join our team in our London office. You will design, build, and maintain data pipelines transforming data from hundreds of thousands of property listings every day and use AI and machine learning to enhance our data.

Key Responsibilities:

• Maintain and improve our data pipeline of live properties.


• Manage PostgreSQL databases.


• Use data processing tools to transform data from hundreds of different sources.


• Use AI and Machine Learning to enrich our data.


• Build data ingestion pipelines from scratch from a variety of different data sources.

Required qualifications:

• Proficient in Python, SQL, pandas with a good general understanding of data engineering technologies.


• Understanding of ETL processes and data warehousing.


• Strong problem-solving and communication skills.


• Minimum 2 years relevant professional experience.


• Able to come into our central London office 5 days per week.

Preferred Qualifications:

• Comfortable with learning and applying new & unfamiliar technologies to ensure best solution is implemented.


• Experience with AWS services (S3, RDS, Lambda, EC2, Athena etc.).


• Experience with machine learning frameworks.


• Experience with data orchestration tools (Airflow, Dagster etc.)