GCP Data Platform

TechDigital Group; San Jose (CA); Vor Ort; USD 140.000 - 190.000; Vollzeit

Zusammenfassung

TechDigital Group in San Jose, CA seeks a Cloud Platform Engineer to administer and optimize Google Cloud environments with a focus on Composer/Airflow, Dataflow, and Dataproc. You will implement infrastructure as code with Terraform, publish Terraform modules to Terraform Enterprise, and manage CI/CD pipelines using Jenkins.

Additionally, you will enforce least-privilege IAM policies, monitor cloud usage and costs, and ensure security and reliability across the platform while documenting

Qualifikationen

• Experience designing and implementing infrastructure as code with Terraform.
• Hands-on experience managing GCP Composer/Airflow environments.
• Proficiency in CI/CD tooling (Jenkins) and automation across cloud platforms.

Aufgaben

• Administer, configure, maintain and upgrade GCP Composer/Airflow environments.
• Design and implement infrastructure as code using Terraform for provisioning and managing GCP resources.
• Manage CI/CD pipelines using Jenkins and/or other CI/CD tools.

Kenntnisse

Problem solving

Troubleshooting

Tools

Terraform

GCP Composer

Cloud Dataflow

Cloud Dataproc

BigQuery

GCS

Cloud Functions

Cloud Run

Pub/Sub

Cloud SQL

Secret Manager

Cloud Monitoring

Cloud Logging

Jobbeschreibung

Job Description

• Administer, configure, maintain and upgrade GCP Composer (Apache Airflow) environment ( Mandatory )
• Design and implement infrastructure as code using Terraform for provisioning and managing GCP resources. (Mandatory)
• Able to create Terraform modules and publish same in TFE (Terraform Enterprise )
• DAG deployment, Troubleshooting, performance tuning, and operational support.
• Support and optimize Dataflow and Dataproc workloads, including batch and streaming data processing pipelines.
• Maintain GCP infrastructure including services like:
  • GCP Composer (Apache Airflow)
  • Google Cloud Storage (GCS)
  • BigQuery (BQ)
  • Cloud Dataflow /Apache Beam
  • Cloud Dataproc
  • Cloud Functions --- (Mandatory)
  • Cloud Run
  • Pub/Sub
  • Cloud SQL
  • Secret Manager
  • Logging & Monitoring
• Monitor GCP usage and quotas, and proactively prevent quota-related issues.
• Set up and manage alerting by Creating Alert policies in GCP Monitoring . (Mandatory)
• Manage CI/CD pipelines using Jenkins and/or using other CI/CD tools.(Mandatory)
• Able to request Identity and Access Management (IAM) permissions, service accounts, and security policies following least-privilege principles.
• Ensure security, compliance, and cost-efficiency across the cloud environment.
• Monitor cloud infrastructure health, performance, and resource utilization using Cloud Monitoring, Cloud Logging, Error Reporting, and Cloud Trace.
• Develop automation scripts and operational tools using Python and/or Java to improve platform efficiency and reduce manual effort.
• Create and maintain operational documentation, runbooks, architecture diagrams, standard operating procedures (SOPs), and platform best practices.
• Excellent problem-solving and troubleshooting skills.

Nice to Have Skills:

• Experience with CDC platforms such as Striim
• Experience with ServiceNow integrations and operational workflows
• Knowledge of FinOps, cloud cost optimization, and governance frameworks
• Networking components understanding including:
  • VPCs and Shared VPCs
  • VPC Connectors
  • Firewall Rules
  • CIDR Ranges

Exposure to healthcare domain