Data Engineer - BigQuery

Nezda Technologies, Inc.; Metro Manila; On-site; PHP 1,339,000 - 2,344,000; Full time

Job summary

Nezda Technologies, Inc. is seeking a Senior Data Engineer to design, build, and optimize our data platform on Google Cloud Platform (GCP). You will develop robust batch and streaming pipelines, handle diverse data ingestion, and ensure data ecosystem reliability using BigQuery and Cloud Spanner.

The role emphasizes SQL proficiency and Python/Java for Apache Beam/Airflow tasks, with a focus on monitoring, tuning, and scalable data storage. Hybrid work model with Metro Manila & Cebu locations.

Qualifications

• Experience designing data platforms on Google Cloud Platform (GCP).
• Proven ability to build and optimize batch and streaming data pipelines.
• Strong SQL; proficient in Python or Java for data engineering tasks.

Responsibilities

• Design, build, and maintain scalable batch and streaming data pipelines on GCP (Dataflow/Beam).
• Ingest data from APIs, databases, and various file formats.
• Orchestrate workflows with Cloud Composer / Apache Airflow and monitor jobs.

Skills

SQL

Python

Java

Data modeling

ETL/ELT

Monitoring

Tools

Dataflow

Cloud Composer

BigQuery

Pub/Sub

Cloud Spanner

Apache Beam

Airflow

Job description

Position: Data Engineer - BigQuery

Work Setup: Hybrid (3X RTO)

Location: Metro Manila & Cebue

Summary:

We are seeking a highly skilled Senior Data Engineer to design, build, and optimize our next-generation data platform on Google Cloud Platform (GCP). In this role, you will be the backbone of our data architecture, responsible for developing robust batch and streaming data pipelines, handling complex file ingestions, and ensuring the absolute reliability of our data ecosystem.

Roles & Responsibilities

• Pipeline Development: Design, develop, and maintain scalable, high-performance batch and streaming data pipelines using GCP Dataflow (Apache Beam).
• Data Ingestion: Build robust ingestion frameworks to consume data from a variety of sources, including API endpoints, databases, and structured/unstructured file formats.
• Orchestration: Author, schedule, and monitor complex data workflows using Cloud Composer / Apache Airflow.
• Data Warehousing & Storage: Optimize data storage and retrieval in BigQuery and manage transactional data integrity within Cloud Spanner.
• Monitoring & Troubleshooting: Proactively monitor pipeline health, diagnose bottlenecks, and troubleshoot data quality or performance issues in production.
• Performance Tuning: Write and optimize complex, highly efficient SQL queries and manage schema evolutions.

Required Skills

GCP Ecosystem

• Strong, hands-on experience with Dataflow, Cloud Composer, and BigQuery. Working knowledge of Cloud Spanner is highly desirable.

Data Streaming

• Proven experience handling both batch and real-time/streaming data processing paradigms (e.g., Pub/Sub to Dataflow).

Languages

• Mastery of SQL (analytical and transactional) and proficiency in Python or Java (for Apache Beam/Airflow).

Operations

• Strong experience in monitoring, logging, and alerting (e.g., GCP Cloud Monitoring/Cloud Logging) and standard debugging practices.

Data Engineering Fundamentals

• Solid understanding of data modeling, ETL/ELT processes, and handling various file formats (Avro, Parquet, JSON, CSV).