GCP Cloud Engineer - Data Pipelines & Infra Expert

TechDigital Group; St. Louis (MO); On-site; USD 90,000 - 120,000; Full time

Job summary

A technology services firm is seeking a skilled cloud developer to manage GCP Composer environments and optimize data ingestion pipelines. The role includes enhancing Dataflow and Dataproc processes, managing Airflow DAGs, and implementing infrastructure as code with Terraform. The ideal candidate is proficient in Java or Python, has strong troubleshooting skills, and communicates effectively with both DevOps and development teams. Comprehensive knowledge of GCP services and tools is essential.

Qualifications

• Proficient in either Java or Python programming languages.
• Excellent problem‑solving and troubleshooting skills.
• Strong communication and documentation abilities.

Responsibilities

• Fix and enhance Dataflow and Dataproc data ingestion pipelines.
• Manage Airflow DAGs and troubleshoot Airflow issues.
• Monitor and optimize Composer performance.

Skills

GCP Composer (Apache Airflow)

Docker

Java

Python

Spring Boot

Terraform

Angular

IAM and service account permissions

Tools

Jenkins

Google Cloud Storage (GCS)

BigQuery (BQ)

Cloud Dataflow (Apache Beam)

Cloud Dataproc

Cloud Functions

Cloud Run

Stackdriver (Cloud Monitoring & Logging)

Job description

A technology services firm is seeking a skilled cloud developer to manage GCP Composer environments and optimize data ingestion pipelines. The role includes enhancing Dataflow and Dataproc processes, managing Airflow DAGs, and implementing infrastructure as code with Terraform. The ideal candidate is proficient in Java or Python, has strong troubleshooting skills, and communicates effectively with both DevOps and development teams. Comprehensive knowledge of GCP services and tools is essential.