Platform Engineer

avacone Switzerland; Greater London; Hybrid; GBP 85,000 - 130,000; Full time

Job summary

Cloudera, Elastic Stack & Dataiku is seeking an experienced Platform Engineer to support, maintain and advance our data platform environment. You will collaborate with data engineers, scientists and stakeholders to deliver scalable analytics solutions.

You will design pipelines, dashboards and integrations, automate deployments, and support ML model deployment in Dataiku, while ensuring robust, secure operations and clear documentation.

Qualifications

• Experience with big data platforms and data pipelines.
• Hands-on development with Python/Scala and REST APIs.
• CI/CD, automation and platform operations experience.

Responsibilities

• Support, maintain and optimise Cloudera, Elastic Stack and Dataiku platforms.
• Design and develop data pipelines, workflows, dashboards and platform integrations.
• Build streaming solutions using Apache Kafka.
• Develop scripts and integrations using Python/Scala and REST APIs.
• Automate ETL and data ingestion with NiFi and Logstash.
• Create Kibana dashboards, reports and search solutions.
• Support ML model deployment in Dataiku.
• Automate deployments with Ansible, Terraform or similar.
• Implement CI/CD with GitHub Actions or Jenkins.
• Troubleshoot platform, workflow and integration issues.
• Create reusable scripts, templates and technical docs.
• Provide guidance and training to users and teams.
• Occasional weekend/public holiday work for system changes.

Skills

Platform engineering

Python

Scala

REST APIs

CI/CD

Troubleshooting

Communication

English proficiency

Tools

GitHub Actions

Jenkins

Terraform

Ansible

NiFi

Logstash

Kibana

Beats

Dataiku DSS

Job description

Cloudera, Elastic Stack & Dataiku

Position Overview

We are looking for an experienced Platform Engineer to support, maintain and further develop our Cloudera, Elastic Stack and Dataiku platform environment. The role combines platform operations, solution development, automation and user support. You will work closely with data engineers, data scientists, analysts and business stakeholders to deliver stable, scalable and efficient data and analytics solutions.

Key Responsibilities

• Support, maintain and optimise Cloudera, Elastic Stack and Dataiku platforms
• Design and develop data pipelines, workflows, dashboards and platform integrations
• Build and support real-time streaming solutions using Apache Kafka
• Develop custom scripts, plugins and integrations using Python, Scala and REST APIs
• Design and automate ETL and data ingestion processes using tools such as Apache NiFi and Logstash
• Create Kibana dashboards, reports, queries and search solutions
• Support the deployment and operationalisation of machine learning models in Dataiku
• Automate platform deployments and operations using Ansible, Terraform or similar tools
• Implement CI/CD pipelines using GitHub Actions, Jenkins or comparable technologies
• Troubleshoot platform, workflow and integration issues
• Create reusable scripts, templates, libraries and technical documentation
• Provide guidance, training and platform support to users and internal teams
• Support occasional weekend or public holiday activities related to system changes or testing

Requirements

Required Skills and Experience

• Strong experience with at least two of the following platforms:
  • Cloudera
  • Elastic Stack
  • Dataiku DSS
• Good knowledge of Hadoop ecosystem technologies such as HDFS, YARN, Hive, Spark and Kafka
• Experience with Elasticsearch, Logstash, Kibana and Beats
• Experience designing data pipelines, ETL processes and streaming solutions
• Programming or scripting experience with Python and/or Scala
• Experience integrating platforms and external systems through REST APIs
• Good knowledge of GitHub, branching strategies, pull requests and code reviews
• Experience with CI/CD, infrastructure automation and deployment processes
• Understanding of clean code, secure coding and documentation standards
• Strong troubleshooting, analytical and communication skills
• Ability to translate business requirements into practical technical solutions
• Professional English skills

Preferred Qualifications

• Experience with machine learning deployment and data science platforms
• Knowledge of Apache NiFi, Terraform, Ansible or Puppet
• Experience working in enterprise-scale data and analytics environments
• Familiarity with agile and DevOps working practices

General Information

• Workload: 100%
• Start Date: As soon as possible
• Location: Flexible, with availability during Swiss working hours

Expected Outcomes

• Stable and well-maintained platform environments
• Reliable delivery of data pipelines, dashboards and analytics solutions
• Automated and optimised platform operations
• Effective user support and knowledge transfer
• Clear, reusable and up-to-date technical documentation