GCP Data Platform Engineer: Airflow, Dataflow & BigQuery

TechDigital Group; San Jose (CA); On-site; USD 140,000 - 190,000; Full time

Job summary

TechDigital Group in San Jose, CA seeks a Cloud Platform Engineer to administer and optimize Google Cloud environments with a focus on Composer/Airflow, Dataflow, and Dataproc. You will implement infrastructure as code with Terraform, publish Terraform modules to Terraform Enterprise, and manage CI/CD pipelines using Jenkins.

Additionally, you will enforce least-privilege IAM policies, monitor cloud usage and costs, and ensure security and reliability across the platform while documenting

Qualifications

• Experience designing and implementing infrastructure as code with Terraform.
• Hands-on experience managing GCP Composer/Airflow environments.
• Proficiency in CI/CD tooling (Jenkins) and automation across cloud platforms.

Responsibilities

• Administer, configure, maintain and upgrade GCP Composer/Airflow environments.
• Design and implement infrastructure as code using Terraform for provisioning and managing GCP resources.
• Manage CI/CD pipelines using Jenkins and/or other CI/CD tools.

Skills

Problem solving

Troubleshooting

Tools

Terraform

GCP Composer

Cloud Dataflow

Cloud Dataproc

BigQuery

GCS

Cloud Functions

Cloud Run

Pub/Sub

Cloud SQL

Secret Manager

Cloud Monitoring

Cloud Logging

Job description

TechDigital Group in San Jose, CA seeks a Cloud Platform Engineer to administer and optimize Google Cloud environments with a focus on Composer/Airflow, Dataflow, and Dataproc. You will implement infrastructure as code with Terraform, publish Terraform modules to Terraform Enterprise, and manage CI/CD pipelines using Jenkins.

Additionally, you will enforce least-privilege IAM policies, monitor cloud usage and costs, and ensure security and reliability across the platform while documenting