GCP Dataflow/Dataproc Engineer (Infra & CI/CD)

TechDigital Group; Virginia (MN); On-site; USD 80,000 - 110,000; Full time

Job summary

TechDigital Group is seeking a talented individual to enhance data ingestion pipelines using GCP services. The ideal candidate will manage Airflow workflows, optimize performance, and ensure security across cloud applications.

Responsibilities include working with Docker and Terraform to provision resources, collaborating closely with DevOps teams, and designing Springboot APIs. Proficiency in Java or Python and knowledge of CI/CD are essential for success in this role.

Qualifications

• Good understanding of GCP Composer for troubleshooting and performance monitoring.
• Experience in managing Airflow-related issues and optimizing pipeline performance.
• Proficient in designing infrastructure as code and managing GCP resources efficiently.

Responsibilities

• Design and implement data ingestion pipelines using GCP services.
• Collaborate with teams to manage CI/CD processes effectively.
• Ensure cloud environment security, compliance, and cost efficiency.

Skills

GCP Composer (Apache Airflow)

DB design

Springboot APIs

Java

Python

Terraform

Docker

Angular

CI/CD pipelines

Tools

Stackdriver (Cloud Monitoring & Logging)

Jenkins

Job description

TechDigital Group is seeking a talented individual to enhance data ingestion pipelines using GCP services. The ideal candidate will manage Airflow workflows, optimize performance, and ensure security across cloud applications.

Responsibilities include working with Docker and Terraform to provision resources, collaborating closely with DevOps teams, and designing Springboot APIs. Proficiency in Java or Python and knowledge of CI/CD are essential for success in this role.