Platform Data Engineer

Mars Food US, LLC; City Of London; On-site; GBP 60,000 - 80,000; Full time

Job summary

A leading food company in the United Kingdom seeks a Data Platform Engineer to design and manage their Azure cloud data platform. Candidates should have strong experience with Terraform, DevOps, and programming in Python. Responsibilities include developing data tools, collaborating with data teams, and utilising Infrastructure as Code practices to ensure efficient resource deployment.

Qualifications

• Proven experience in data platform engineering, building and optimising data infrastructure.
• Strong experience in Terraform, Infrastructure as Code, and DevOps pipelines.
• Familiarity with containerisation and orchestration technologies.

Responsibilities

• Design and maintain data platforms on Azure cloud, ensuring performance and scalability.
• Develop and implement data platform tools to support organisational needs.
• Collaborate with data teams to align infrastructure with business objectives.
• Utilise Infrastructure as Code for efficient resource management.

Skills

Data platform engineering

Terraform

Infrastructure as Code

DevOps pipelines

Bash scripting

PowerShell scripting

Proficiency in Python

Azure cloud services

Communication skills

Education

Bachelor's or master's degree in computer science or related field

Tools

Docker

Kubernetes

Hadoop

Spark

AWS

Google Cloud

Snowflake

Job description

Job Description

What are we looking for?

Essential Requirements

• Proven experience in data platform engineering, including building, developing, and optimising data infrastructure.
• Proficient primarily in Terraform, with strong experience in Infrastructure as Code, DevOps pipelines, and scripting using Bash and PowerShell.
• Proficiency in Python.
• Expertise in Azure cloud services and infrastructure as code (e.g., Azure Resource Manager templates, Terraform) and DevOps Pipelines such as Azure DevOps Pipelines or GitHub Actions.
• Excellent communication skills being able to explain complex platform concepts to both data engineers and business users.

Nice to Haves

• Bachelor's or master's degree in computer science, Information Technology, or a related field
• Experience with containerisation and orchestration technologies (e.g., Docker, Kubernetes).
• Knowledge of big data technologies and frameworks (e.g., Hadoop, Spark).
• Familiarity with other cloud platforms (e.g. AWS, Google Cloud) and PaaS providers (e.g. Snowflake)
• Knowledge of Inner or Open Source paradigm and way of working
• Knowledge of Cloud (Azure) Networking and Security Standards

Key Responsibilities

• Data Platform Engineering: Contribute to designing, developing, and maintaining our data platform and core toolsets on the Azure cloud. Ensure the platform is highly available, scalable, and performs optimally.
• Data Tools and Frameworks Development: Contribute to developing and implementing data platform tools and frameworks on Azure to support the organisation's data needs. Collaborate with other teams to align data infrastructure with business objectives.
• Collaborate and support platform users: Work alongside other data teams and business users to help understand their requirements of the platform and support in a third-line capability as and when needed.
• Infrastructure as Code: Utilise Azure infrastructure as code practices to automate the provisioning and management of data platform resources. Ensure consistency, repeatability, and efficiency in infrastructure deployment.
• Technology Evaluation: Stay up-to-date with emerging technologies and industry best practices in data engineering and Azure cloud services. Evaluate and recommend new tools and technologies to enhance the data platform's capabilities and efficiency.