Ingestion Engineer

Mars, Incorporated and its Affiliates; Greater London; On-site; GBP 50,000 - 70,000; Full time

Job summary

A leading global food company is seeking an Ingestion Engineer to join their DDF team in Greater London. You will engage in hands-on development using technologies like Databricks and Python, working on data models and pipelines in a cloud environment. The role emphasizes collaboration with analysts and data scientists while supporting the Pet Nutrition data platform. Join us to contribute to innovative data solutions and grow your skills in a dynamic setting.

Qualifications

• Experience with Databricks, Python, and data ingestion technologies.
• Knowledge of data pipelines and cloud environments.
• Familiarity with DevOps and CI/CD methodologies.

Responsibilities

• Develop and manage data models and pipelines in a cloud environment.
• Collaborate with team members on data engineering best practices.
• Support the development of data solutions for the Pet Nutrition platform.

Skills

Databricks

Python

PySpark

SQL

Azure Functions

Tools

Delta Lake

Logic Apps

Django

ReactJs

Job description

Job Description

As an Ingestion Engineer in the DDF team, your key responsibilities are as follows:

1. Technical Proficiency

• Collaborate in hands‑on development using Databricks, Python, PySpark, ADF, and other relevant technologies to ingest data for downstream consumption.
• Assist in engineering and managing data models and pipelines within a cloud environment, utilizing technologies like Databricks, Spark, Delta Lake, and SQL.
• Contribute to the maintenance and enhancement of our progressive tech stack, which includes Python, PySpark, Logic Apps, Azure Functions, ADLS, Django, and ReactJs.
• Support the implementation of DevOps and CI/CD methodologies to foster agile collaboration and contribute to building robust data solutions.
• Develop code that adheres to high‑quality standards, promoting a scalable and maintainable data platform.

2. Learning and Growth; Contribution to Solutions

• Collaborate with the team to learn and apply the best practices in data engineering.
• Actively participate in engineering projects, gaining experience in developing high‑quality, scalable, and sustainable data solutions.
• Stay updated with emerging technologies and trends in data engineering, contributing to the team's knowledge base by sharing insights and ideas.
• Assist in the development of data solutions within the Pet Nutrition data platform, working on challenging aspects under the guidance of senior team members.
• Contribute to the management of data from various divisions to generate valuable data assets related to pets and pet owners.
• Support the maintenance of a semantic and intelligent data layer to contribute to the comprehensive leadership of the data solution within the environment.

3. Collaboration and Communication

• Collaborate closely with analysts, data scientists, and team members to understand their requirements and assist in translating them into actionable data solutions.
• Maintain effective communication with the Data Ingestion Lead, actively participating in team discussions and sharing ideas to improve platform excellence.

#TBDDT