Senior Data Platform Engineer (Remote)

WA Consultants Ltd; United States; Remote; USD 100,000 - 150,000; Full time

Job summary

A leading company is seeking a highly skilled Data Platform Engineer for a full-time, fully remote contract. This role focuses on supporting a distributed streaming platform, designing and optimizing real-time scalable data pipelines primarily using Kafka and Kubernetes in cloud environments like GCP or Azure. The ideal candidate will have hands-on experience with data platforms and cloud infrastructure, working collaboratively within cross-functional teams. Applicants should possess excellent skills in data streaming technologies and infrastructure as code.

Qualifications

• Strong hands-on experience with Apache Kafka in production.
• Proficiency with Kubernetes (GKE, AKS, or self-managed).
• Solid understanding of cloud infrastructure in GCP and/or Azure.

Responsibilities

• Design, deploy, and manage Kafka clusters in GCP or Azure.
• Operate and maintain Kafka on Kubernetes using Helm.
• Collaborate in building data pipelines and integrating with microservices.

Skills

Apache Kafka

Kubernetes

Cloud infrastructure

Python

Java

CI/CD pipelines

Monitoring tools

Job description

Required for a fulltime, six moth, fully remote contract, a highly skilled Data Platform Engineer to support our distributed streaming platform for handling real-time streaming data. This role is ideal for someone with deep expertise in Kafka, Kubernetes and microservices, also cloud platforms like Google Cloud Platform (GCP) or Microsoft Azure. You’ll play a key role in designing, deploying, and optimizing scalable, real-time data pipelines in a cloud-native environment.

Key Responsibilities:

* Design, deploy, and manage Kafka clusters in GCP or Azure environments
* Operate and maintain Kafka on Kubernetes using Helm, Operators, or custom configurations
* Collaborate with cross-functional teams in building data pipelines, integrating with Kafka and building microservices
* Monitor, troubleshoot, and optimize Kafka performance and reliability
* Automate infrastructure and deployment processes using Infrastructure-as-Code tools
* Ensure security, compliance, and high availability of Kafka systems

Qualifications:

* Strong hands-on experience with Apache Kafka in production environments
* Proficiency with Kubernetes (GKE, AKS, or self-managed clusters)
* Solid understanding of cloud infrastructure in GCP and/or Azure
* Experience with CI/CD pipelines and DevOps practices
* Strong programming skills in Python, Java, or similar languages
* Familiarity with monitoring tools (e.g., Prometheus, Grafana) and logging systems
* Excellent communication and documentation skills

Preferred Qualifications:

* Experience with Confluent Platform or Confluent Cloud
* Knowledge of Terraform, Helm, or other IaC tools
* Background in stream processing (Kafka Streams, RabbitMQ Streams, Flink, RocksDB, etc.)
* Certifications in GCP, Azure, or Kubernetes (CKA/CKAD)
* Experience working in agile, cross-functional teams

To arrange a Teams-based interview, please email in the first instance, your CV to [e-mail]

WA Consultants is an Employment Business and an Employment Agency as described within The Conduct of Employment Agencies and Employment Businesses Regulations 2003.