Role: GCP Data Engineer
Location: Phoenix AZ, New York, Sunrise FL, Charlotte, NC (This position required face 2 face interview but this will be sponsred by client)
Hire Mode: W2 Contract
Job Summary
We are looking for a skilled and analytical GCP Data Engineer to design, build, and optimize our data infrastructure. In this role, you will create robust batch and streaming data pipelines, manage our cloud data warehouse, and ensure data quality, security, and scalability. You will collaborate closely with Data Scientists, Business Analysts, and DevOps teams to translate complex business requirements into high-performing technical solutions on the Google Cloud Platform (GCP).
Core Responsibilities
- Pipeline Development: Design, build, and maintain scalable batch and real-time streaming data pipelines using GCP-native services like Cloud Dataflow (Apache Beam) and Cloud Pub/Sub.
- Data Warehousing and Architecture: Implement and optimize data storage and warehouse architectures using BigQuery. Write highly complex, optimized SQL queries and manage data partitioning/clustering to maintain performance and cost efficiency.
- Orchestration: Develop, deploy, and manage workflow automation and DAGs using Cloud Composer (Apache Airflow).
- Data Modeling: Transform raw data into structured, clean datasets ready for analysis, utilizing tools like dbt (Data Build Tool) and dimensional modeling concepts.
- Operations and Monitoring: Monitor pipeline performance, troubleshoot bottlenecks, and implement proactive alerting using Cloud Monitoring and Cloud Logging.
- Security and Compliance: Ensure all data pipelines and storage solutions adhere to governance policies, handling access controls via IAM and safeguarding sensitive data.
- Collaboration: Work alongside cross-functional engineering, product, and data science teams to deliver actionable data insights and support analytics initiatives.
Required Qualifications
- Education: Bachelor degree in Computer Science, Information Systems, Engineering, or a related technical field.
- Experience: 3 to 5 plus years of dedicated experience in a Data Engineering role, with a strong track record of building production-grade environments on Google Cloud Platform.
- Advanced SQL: Exceptional capability in writing, tuning, and debugging analytical SQL queries.
- Programming Skills: Proficiency in Python (highly preferred) or Java/Scala for data manipulation and automation scripting.
- GCP Ecosystem Expertise: Hands-on experience with:
- Storage and Analytics: BigQuery, Cloud Storage, Cloud SQL
- Compute and Ingestion: Cloud Dataflow, Cloud Dataproc (Spark/Hadoop), Cloud Pub/Sub
- Orchestration: Cloud Composer / Apache Airflow
- DevOps and CI/CD: Familiarity with Git version control and deploying cloud infrastructure using infrastructure-as-code tools like Terraform.