Senior Data Platform Engineer

Tookitaki; Manila; On-site; PHP 1,800,000 - 2,400,000; Full time

Job summary

Tookitaki seeks a battle-tested Data Platform Engineer to build, operate, and evolve large-scale data platforms across Cloudera on-premises and cloud environments, plus AWS/Kubernetes architectures. You will work where distributed compute, storage, and production pipelines meet, tracing workloads from application to Spark or Flink.

Ideal candidates have strong Cloudera production experience, Hadoop/Hive expertise, and solid automation skills with Python, Bash, Terraform, and Ansible.

Qualifications

• Extensive hands-on experience administering Cloudera platforms in production.
• Experience with on-premises Cloudera and cloud-based Cloudera deployments.
• Deep knowledge of Hadoop, HDFS, Hive, YARN, Spark, and the wider distributed-data ecosystem.
• Experience building or operating production workloads using Apache Flink.
• Strong understanding of distributed data processing, including partitioning, shuffling, serialization, checkpointing, and failure recovery.
• Experience transforming large datasets using joins, aggregations, filtering, sorting, and schema evolution.
• Proficiency with Parquet, Avro, JSON, CSV, compression formats, and serialization tradeoffs.
• Experience designing data layouts for query performance, ingestion throughput, retention, and cost.
• Strong AWS experience with EMR, S3, EC2, EKS, IAM, CloudWatch, and KMS.
• Strong Kubernetes experience with operators, controllers, Helm, scheduling, storage, networking, and troubleshooting.
• Experience migrating data platforms from on-premises or VM-based environments into AWS and Kubernetes.
• Ability to troubleshoot Linux, JVM, networking, storage, DNS, certificates, and resource-management issues.
• Experience with observability platforms and correlating infrastructure symptoms with data-pipeline failures.
• Ability to automate operational work using Python, Bash, Terraform, Ansible, or equivalent tools.
• Strong judgment around production changes, data integrity, access control, rollback, and recovery.

Responsibilities

• Build and operate production data platforms across Cloudera on premises, Cloudera cloud environments, AWS EMR, and Kubernetes.
• Design batch and streaming pipelines using Spark, Flink, Hive, and related technologies.
• Transform large datasets through filtering, sorting, joining, aggregation, partitioning, enrichment, and restructuring.
• Work with Parquet, Avro, JSON, CSV, and other delimited or semi-structured formats.
• Design storage, partitioning, compression, retention, and lifecycle strategies across HDFS, Hive, and object storage.
• Design and maintain Hive schemas, tables, partitions, metadata, and data models.
• Administer Cloudera clusters, including installation, upgrades, configuration, scaling, patching, security, backup, and recovery.
• Troubleshoot unhealthy services, failed jobs, resource contention, data skew, small-file problems, metadata issues, and storage bottlenecks.
• Tune Spark and Flink workloads for memory, CPU, parallelism, shuffle behavior, checkpointing, and recovery.
• Operate AWS services such as EMR, S3, IAM, EC2, EKS, CloudWatch, KMS, and supporting networking services.
• Deploy and operate data workloads on Kubernetes using operators, Helm, custom resources, and GitOps-based delivery.
• Help migrate workloads from physical or virtual machines and EC2 into AWS and Kubernetes-based platforms.
• Separate compute from storage where appropriate while accounting for performance, resilience, security, and cost.
• Build monitoring, alerting, capacity management, and operational runbooks for critical data services.
• Automate platform provisioning and configuration using Terraform, Ansible, scripting, and CI/CD.
• Support production incidents involving failed pipelines, delayed data, cluster degradation, storage pressure, or infrastructure failure.
• Work with data engineering, infrastructure, security, and application teams to resolve problems across ownership boundaries.

Skills

Python

Bash

Terraform

Ansible

Spark

Flink

Hive

HDFS

Kubernetes

AWS

Cloudera

Linux

Observability

Data pipelines

Tools

Cloudera

Kubernetes

Spark

Flink

Hive

HDFS

AWS EMR

Terraform

Ansible

Job description

The mission

We are looking for a battle-tested Data Platform Engineer who can build, operate, troubleshoot, and evolve large-scale data platforms across on-premises Cloudera environments and cloud-native AWS/Kubernetes architectures.

This is not a dashboard or SQL-only role. You will work where distributed compute, storage, networking, Kubernetes, and production data pipelines meet. You must be comfortable tracing a failed workload from the application layer through Spark or Flink,

Kubernetes operators, HDFS/Hive, infrastructure, and AWS services.

The goal is to help us evolve safely from on-premises, VM, and EC2-based platforms into resilient, observable, cloud-native data systems.

What you’ll do

• - Build and operate production data platforms across Cloudera on premises, Cloudera cloud environments, AWS EMR, and Kubernetes.
• - Design batch and streaming pipelines using Spark, Flink, Hive, and related technologies.
• - Transform large datasets through filtering, sorting, joining, aggregation, partitioning, enrichment, and restructuring.
• - Work with Parquet, Avro, JSON, CSV, and other delimited or semi-structured formats.
• - Design storage, partitioning, compression, retention, and lifecycle strategies across HDFS, Hive, and object storage.
• - Design and maintain Hive schemas, tables, partitions, metadata, and data models.
• - Administer Cloudera clusters, including installation, upgrades, configuration, scaling, patching, security, backup, and recovery.
• - Troubleshoot unhealthy services, failed jobs, resource contention, data skew, small-file problems, metadata issues, and storage bottlenecks.
• - Tune Spark and Flink workloads for memory, CPU, parallelism, shuffle behavior, checkpointing, and recovery.
• - Operate AWS services such as EMR, S3, IAM, EC2, EKS, CloudWatch, KMS, and supporting networking services.
• - Deploy and operate data workloads on Kubernetes using operators, Helm, custom resources, and GitOps-based delivery.
• - Help migrate workloads from physical or virtual machines and EC2 into AWS and Kubernetes-based platforms.
• - Separate compute from storage where appropriate while accounting for performance, resilience, security, and cost.
• - Build monitoring, alerting, capacity management, and operational runbooks for critical data services.
• - Automate platform provisioning and configuration using Terraform, Ansible, scripting, and CI/CD.
• - Support production incidents involving failed pipelines, delayed data, cluster degradation, storage pressure, or infrastructure failure.
• - Work with data engineering, infrastructure, security, and application teams to resolve problems across ownership boundaries.

What we’re looking for

• - Strong hands-on experience administering Cloudera platforms in production.
• - Experience with both on-premises Cloudera and cloud-based Cloudera deployments.
• - Deep working knowledge of Hadoop, HDFS, Hive, YARN, Spark, and the wider distributed-data ecosystem.
• - Experience building or operating production workloads using Apache Flink.
• - Strong understanding of distributed data processing, including partitioning, shuffling, serialization, checkpointing, and failure recovery.
• - Experience transforming large datasets using joins, aggregations, filtering, sorting, and schema evolution.
• - Practical knowledge of Parquet, Avro, JSON, CSV, compression formats, and serialization tradeoffs.
• - Experience designing data layouts for query performance, ingestion throughput, retention, and cost.
• - Strong AWS experience, particularly with EMR, S3, EC2, EKS, IAM, CloudWatch, and KMS.
• - Strong Kubernetes experience, including operators, controllers, Helm, scheduling, storage, networking, and workload troubleshooting.
• - Experience migrating data platforms from on-premises or VM-based environments into AWS and Kubernetes.
• - Ability to troubleshoot Linux, JVM, networking, storage, DNS, certificates, and resource-management issues.
• - Experience with observability platforms and the ability to correlate infrastructure symptoms with data-pipeline failures.
• - Ability to automate operational work using Python, Bash, Terraform, Ansible, or equivalent tools.
• - Strong judgment around production changes, data integrity, access control, rollback, and recovery.

Production scenarios you should be able to handle

• - A Spark job that ran in 40 minutes yesterday now takes four hours.
• - A join creates severe data skew and repeatedly exhausts executor memory.
• - HDFS is approaching capacity while NameNode health is degrading.
• - Hive queries return incomplete results because partitions or metadata are inconsistent.
• - A Flink job repeatedly fails after checkpoint recovery.
• - An EMR workload is reliable but significantly more expensive than expected.
• - A Kubernetes operator reports success while the underlying data workload is unhealthy.
• - A migrated workload behaves differently on S3 than it did on HDFS.
• - A certificate, Kerberos, IAM, DNS, or network problem presents as an application failure.
• - A critical pipeline misses its SLA and ownership is unclear across platform and data teams.

Certifications

Relevant certifications are useful, particularly:

• - Cloudera Administrator certification or equivalent production experience
• - AWS data, analytics, or architecture certifications
• - Kubernetes certifications such as CKA or CKAD

What success looks like

• - Data pipelines meet their reliability and processing-time objectives.
• - Platform failures are detected before downstream consumers report them.
• - Incidents move quickly from symptoms to an evidence-backed root cause.
• - Cloudera, AWS, and Kubernetes environments are operated through repeatable automation.
• - Migrations preserve data correctness while improving scalability and operability.
• - Storage and compute designs balance performance, resilience, and cost.
• - Data engineers can ship workloads without becoming accidental platform administrators.
• - Operational knowledge becomes monitoring, automation, and runbooks—not tribal memory.

The person we want

You understand that a data platform is a distributed production system, not a collection of product names.

You can move from a Hive execution plan to Spark executor logs, Kubernetes events, HDFS health, S3 behavior, IAM permissions, and network telemetry without losing the thread. You know the architectural differences between on-premises Hadoop and cloud-

native data platforms, including where a lift-and-shift approach will fail.

We need someone who can enter a degraded platform, establish the facts, protect data integrity, restore service, explain the failure chain, and make the system harder to break next time.