Real-Time Data Engineer — High-Throughput Pipelines on AWS

Penbrothers; Mandaluyong; On-site; PHP 1,200,000 - 1,800,000; Full time

Job summary

Penbrothers in Metro Manila seeks a Data Engineer to join a cloud-native data team building large-scale platforms with real-time streaming. The role emphasizes distributed systems, Databricks, and a strong software engineering mindset within AWS.

You will design and operate high-throughput pipelines using Kafka, Spark, Airflow, and Python/Java, focusing on data quality, observability, and scalable APIs. Office-based in Mandaluyong with standard M-F hours.

Qualifications

• Hands-on experience with streaming and real-time data technologies such as Apache Kafka, Spark (Core/Streaming), or RabbitMQ.
• Hands-on experience with Databricks and its ecosystem.
• Proven experience building and maintaining distributed data pipelines using Spark and Apache Airflow.
• Strong programming skills in Python or Java with solid software engineering fundamentals.
• Advanced SQL skills with relational, columnar, and NoSQL databases (MySQL, PostgreSQL, MongoDB, Elasticsearch).
• Experience on AWS cloud ecosystem (e.g., EMR, MSK, Glue, S3).

Responsibilities

• Design, build, and maintain high-throughput, scalable data pipelines within a Databricks ecosystem.
• Develop, optimize, and maintain real-time streaming and complex batch data processing workflows.
• Ensure data quality, reliability, and observability across live data streams through validation and monitoring frameworks.
• Build high-performance data APIs and services for internal and external consumption.
• Troubleshoot and resolve production infrastructure and streaming pipeline issues.
• Participate in code reviews and Agile ceremonies.

Skills

Streaming tech

Databricks

Kafka/Spark/RabbitMQ

Airflow

Python/Java

SQL/NoSQL

Tools

Databricks

Kafka

Spark

Airflow

RabbitMQ

Job description

Penbrothers in Metro Manila seeks a Data Engineer to join a cloud-native data team building large-scale platforms with real-time streaming. The role emphasizes distributed systems, Databricks, and a strong software engineering mindset within AWS.

You will design and operate high-throughput pipelines using Kafka, Spark, Airflow, and Python/Java, focusing on data quality, observability, and scalable APIs. Office-based in Mandaluyong with standard M-F hours.