Options Google Cloud Data Migration Lead

Cloud Hybrid Technologies, LLC; Dallas (TX); On-site; USD 82,656 - 123,984; Full time

Job summary

A leading technology firm is seeking a Google Cloud Data Migration Lead to oversee large-scale data migrations. This role requires an expert in creating scalable data pipelines using Google Cloud technology. The ideal candidate has over 8 years of experience in data engineering, excellent SQL skills, and proficiency in using Google Cloud tools. This long-term contract position offers a flexible remote work environment.

Qualifications

• 8+ years of data engineering experience developing large data pipelines.
• Very strong SQL skills for complex transformation data pipelines.
• Hands-on experience in at least three data migrations to Google Cloud.

Responsibilities

• Serve as Data Migration team leader for a migration to Google Cloud.
• Design and build data pipelines using GCP tools.
• Ensure data quality and integrity throughout the data pipeline.

Skills

Data engineering experience

Strong SQL skills

Data migration methods and tooling

Programming languages (Python)

Tools

Google Cloud Platform

BigQuery

Cloud Dataflow

Cloud Pub/Sub

Terraform

CI/CD tools

Job description

Overview

Options Google Cloud Data Migration Lead

Location: Remote

Duration: 18 Months + Extension

Hourly Rate: Depending on Experience (DOE)

Work Authorization: Location: Remote

Duration: Long term contract

Job Summary

Seeking a Google Cloud data engineer to design, build, and maintain scalable and efficient data processing systems on the Google Cloud platform. This engineer will be responsible for the entire data lifecycle, from ingestion and storage to processing, transformation, and analysis. Their work will enable client organizations to make data-driven decisions by providing clean, high-quality data to business intelligence tools, AI systems, analysts and data scientists.

Responsibilities

• Serve as Data Migration team leader for a large data and application migration to the Google Cloud platform.
• As Data Migration team leader, this individual will be responsible for our team's endto-end data architecture and migration planning to support the migration effort as well as future-state client efforts on the Google Cloud platform.
• As Data Migration team leader, this individual will collaborate closely with the overall Google Cloud migration team leadership, working to deliver a successful application and data migration.
• Design and build data pipelines: Develop and maintain reliable and scalable batch and real-time data pipelines using Google Cloud Platform tools such as Cloud Dataflow (based on Apache Beam), Cloud Pub/Sub, and Cloud Composer (for Apache Airflow).
• Create and manage data storage solutions: Implement data warehousing and data lake solutions using Google Cloud Platform products like BigQuery, Cloud Storage, and other transactional or NoSQL databases such as CloudSQL or Bigtable.
• Ensure data quality and integrity: Develop and enforce procedures for data governance, quality control, and validation throughout the data pipeline to ensure data is accurate and reliable.
• Optimize performance and cost: Monitor data infrastructure and pipelines to identify and resolve performance bottlenecks, ensuring that all data solutions are cost-effective and scalable.
• Collaborate with other teams: Work closely with data scientists, analysts, and business stakeholders to gather requirements and understand data needs, translating them into technical specifications.
• Automate and orchestrate workflows: Automate data processes and manage complex workflows using tools like Cloud Composer.
• Implement security: Design and enforce data security and access controls using Google Cloud Platform Identity and Access Management (IAM) and other best practices.
• Maintain documentation: Create and maintain clear documentation for data pipelines, architecture, and operational procedures.

Required Skills & Qualifications

• 8+ years of data engineering experience developing large data pipelines in very complex environments
• Very Strong SQL skills and ability to build very complex transformation data pipelines using custom ETL framework in Google BigQuery environment
• Very strong understanding of data migration methods and tooling, with hands-on experience in at least three (3) data migrations to Google Cloud

Technical Proficiencies

• Google Cloud Platform: Hands-on experience with key Google Cloud Platform data services is essential, including:
• BigQuery: For data warehousing and analytics.
• Cloud Dataflow: For building and managing data pipelines.
• Cloud Pub/Sub: For real-time messaging and event ingestion.
• DataProc: For running Apache Spark and other open-source frameworks.
• Programming languages: Strong proficiency in programming languages, most commonly Python, is mandatory. Experience with Java or Scala is also preferred.
• SQL expertise: Advanced SQL skills for data analysis, transformation, and optimization within BigQuery and other databases.
• ETL/ELT: Deep knowledge of Extract, Transform, Load (ETL) and Extract, Load, Transform (ELT) processes.
• Infrastructure as Code (IaC): Experience with tools like Terraform for deploying and managing cloud infrastructure.
• CI/CD: Familiarity with continuous integration and continuous deployment (CI/CD) pipelines using tools such as GitHub Actions or Jenkins.
• Data modeling: Understanding of data modeling, data warehousing, and data lake concepts

Equal Employment Opportunity

Cloud Hybrid is an equal opportunity employer inclusive of female, minority, disability and veterans. Hiring, promotion, transfer, compensation, benefits, discipline, termination and all other employment decisions are made without regard to race, color, religion, sex, sexual orientation, gender identity, age, disability, national origin, citizenship/immigration status, veteran status or any other protected status. Cloud Hybrid will not make any posting or employment decision that does not comply with applicable laws relating to labor and employment, equal opportunity, employment eligibility requirements or related matters. Nor will Cloud Hybrid require in a posting or otherwise U.S. citizenship or lawful permanent residency in the U.S. as a condition of employment except as necessary to comply with law, regulation, executive order, or federal, state, or local government contract