Senior GCP Data Engineer & Pipeline Lead

AppLab Systems, Inc; New Jersey; On-site; USD 120,000 - 180,000; Full time

Job summary

AppLab Systems, Inc. seeks a Senior Data Engineer to design, develop, and maintain scalable data pipelines on Google Cloud Platform. You will work with architects, data analysts, and stakeholders to deliver high-quality data solutions in BigQuery and related services.

The role emphasizes ingestion, transformation, orchestration, governance, and cost/performance optimization across enterprise data systems, with a focus on reliable, scalable data models and governance practices.

Qualifications

• Strong experience designing and maintaining scalable data pipelines on GCP.
• Proficiency in Python for data engineering and automation.
• Expertise in BigQuery SQL, SPs, views, and UDFs.
• Experience migrating data warehouses to BigQuery.
• Knowledge of data governance, metadata, and lineage on GCP.
• Experience with Airflow/Cloud Composer.

Responsibilities

• Design and implement end-to-end data pipelines using Cloud Dataflow (Python/Apache Beam) for batch and streaming data.
• Develop, optimize, and maintain BigQuery stored procedures, SQL scripts, and UDFs for transformations.
• Build and manage data orchestration workflows using Cloud Composer (Airflow).
• Establish secure connections to source systems for data ingestion.
• Manage data ingestion workflows from on-premise and cloud sources into GCS and BigQuery.
• Migrate history data from legacy warehouses to BigQuery for performance optimization.
• Design data validation and testing frameworks for data quality across pipelines.
• Implement data governance, including metadata management and access control.
• Collaborate to define scalable data models, views, and semantic layers in BigQuery.
• Troubleshoot pipeline failures and perform root cause analysis.
• Optimize cost and performance of GCP workloads using best practices.

Skills

GCP BigQuery

Dataflow (Python)

Cloud Composer

SQL tuning

Data modeling

ETL/ELT

Airflow

Tools

Snowflake

Teradata

Netezza

Oracle

SQL Server

Job description

AppLab Systems, Inc. seeks a Senior Data Engineer to design, develop, and maintain scalable data pipelines on Google Cloud Platform. You will work with architects, data analysts, and stakeholders to deliver high-quality data solutions in BigQuery and related services.

The role emphasizes ingestion, transformation, orchestration, governance, and cost/performance optimization across enterprise data systems, with a focus on reliable, scalable data models and governance practices.