(Senior) Lead Data Engineer in Staines-upon-Thames - IFS

WorksHub; Staines-upon-Thames; On-site; GBP 50,000 - 75,000; Full time

Job summary

A premier tech company in Staines-upon-Thames is seeking an expert in large-scale data processing with strong programming skills in Python, Scala, C#, or Java. The ideal candidate will have proficiency in tools like Airbyte and Fivetran, along with AI/ML expertise in frameworks such as TensorFlow and PyTorch. This role offers an opportunity to enhance pipeline efficiency using diverse data sources.

Skills

Airbyte

Fivetran

Spark

Dask

Python

Scala

C#

Java

AI/ML

TensorFlow

PyTorch

AutoML

R

Job description

Skills and Expertise

When it comes to working with diverse data sources, proficiency in tools like Airbyte and Fivetran is a must. An expert in large-scale data processing, particularly with Spark or Dask, is highly desired. Strong programming skills in Python, Scala, C#, or Java, along with experience in cloud SDKs and APIs, are key requirements for this role.

In addition to these technical skills, AI/ML expertise plays a crucial role in optimizing pipelines. Familiarity with frameworks such as TensorFlow, PyTorch, and AutoML, as well as programming languages like Python and R, is essential for enhancing pipeline efficiency and effectiveness.