Principal Data Engineer

TN United Kingdom; Staines-upon-Thames; On-site; GBP 60,000 - 100,000; Full time

Job summary

An innovative firm is seeking a Principal Data Engineer to join their dynamic R&D team. This role involves revolutionizing data analytics and AI integration into their cloud solutions, tackling challenges like predictive maintenance and anomaly detection. You'll lead the data engineering efforts, crafting efficient data pipelines and collaborating with cross-functional teams to drive advanced AI solutions. If you are passionate about data and eager to make an impact in the AI space, this is the perfect opportunity to showcase your expertise in a vibrant and forward-thinking environment.

Qualifications

• 7+ years of data engineering experience with scalable solutions.
• Proficient in orchestrating data workflows using Kubernetes.

Responsibilities

• Lead the Data Engineering team in crafting scalable data solutions.
• Collaborate with ML Engineers and Data Scientists to deploy data pipelines.

Skills

Data Engineering

Python

Azure

Data Lakes

Kubernetes

Airflow

Spark

CI/CD Automation

Machine Learning

IoT Data Pipelines

Tools

Airbyte

Fivetran

TensorFlow

PyTorch

Kafka

Terraform

Azure DevOps

Job description

Social network you want to login/join with:

Principal Data Engineer, Staines-upon-Thames

Client: IFS

Location: Staines-upon-Thames, United Kingdom

Job Category: -

EU work permit required: Yes

Job Reference: 569a31107557

Job Views: 10

Posted: 26.04.2025

Expiry Date: 10.06.2025

Job Description

Are you ready to make waves in the world of AI? We're on the hunt for a Principal Data Engineer to join our dynamic global R&D organization. We're looking for someone who brings the heat, fosters seamless collaboration, and is always chasing that next level of excellence.

You'll be at the forefront of infusing cutting-edge advanced analytics and AI into IFS Cloud, revolutionizing Enterprise Resource Planning, Asset Management, and Field Service Management. Get ready to tackle high-stakes challenges like IIoT, predictive maintenance, forecasting, anomaly detection, optimization, and unleashing generative AI. Your data engineer wizardry will power our solutions, crafting efficient data pipelines, expanding our data platform capabilities, and pushing the envelope of data-driven innovation across our product lineup.

Your sharp critical thinking and knack for real-world business dilemmas will be instrumental in orchestrating end-to-end solutions. From spotting opportunities on the horizon to delivering high-performance, scalable data solutions, you'll play a pivotal role in our success.

If you're a maestro of mapping business processes and deciphering complex data, if advanced analytics and AI are your jam, and if you take pride in building top-tier data pipelines for production environments, we want to hear from you.

How Will You Shape the Future?

This role is all about hands-on technical prowess, and we expect you to bring your A-game. You'll be in the driver's seat, working with autonomy, accountability, and technical brilliance. Your mission includes:

• Spotting high-value data opportunities within our IFS offerings, translating raw data into powerful features and reusable data assets.
• Serving as our data expert, guiding us towards the latest and greatest data technology and platform trends. You'll be the guru driving our data platform evolution and providing data project estimates.
• Leading the Data Engineering team in crafting and integrating data projects from the ground up. From framing problems and experimenting with new data sources and tools to the grand finale of data pipeline implementation and deployment. You will ensure scalability and top-tier performance.
• Collaborating with ML Engineers, Data Scientists, Architects, and Product/Program Managers. Together, you'll define, create, deploy, monitor, and document data pipelines to power advanced AI solutions.
• Becoming our data technology evangelist. Get ready to shine on the conference stage, host webinars, and pen compelling white papers and blogs. Share your discoveries with clients and internal stakeholders, offering actionable insights that drive change.

Qualifications

To succeed in this role, you'll need:

• 7+ years of data engineering experience, skilled in scalable solutions like Data Lakes/Lakehouse, Graph and Vector Databases (, ADLS, Elasticsearch, MongoDB, Azure AI search, etc.).
• Proficient in data pipelines across cloud/on-premises, using Azure and other technologies.
• Experienced in orchestrating data workflows and Kubernetes clusters on AKS using Airflow, Kubeflow, Argo, Dagster or similar.
• Skilled with data ingestion tools like Airbyte, Fivetran, etc. for diverse data sources.
• Expert in large-scale data processing with Spark or Dask.
• Strong in Python, Scala, C# or Java, cloud SDKs and APIs.
• AI/ML expertise for pipeline efficiency, familiar with TensorFlow, PyTorch, AutoML, Python/R, and MLOps (MLflow, Kubeflow).
• Solid in DevOps, CI/CD automation with Bitbucket Pipelines, Azure DevOps, GitHub.
• Automate deployment of data pipelines and applications using Bash, PowerShell, or Azure CLI, Terraform, Helm Charts etc.
• Experienced in leveraging Azure AI Search, Elasticsearch, MongoDB or other hybrid/vector stores for content analysis and indexing, with a focus on creating advanced RAG (Retrieval Augmented Generation) applications.
• Proficiency in building IoT data pipelines, encompassing real-time data ingestion, transformation, security, scalability, and seamless integration with IoT platforms.
• Design, develop, and monitor streaming data applications using Kafka and related technologies.

Ready to make your mark? Join us on this exhilarating journey, where you'll be a vital part of our AI revolution. Let's transform the future together!