Job Description: 2 Candidate Submittal Slots, New High Level Policy
Bill Rate $70 - $80
MSP Owner: Rob Finton
Location: Preferred Memphis, TN or Dallas, TX if not any in US - Remote or Onsite/Hybrid
Duration: 6 months
Competencies: 6-8+ years experience
Digital : Google Data Engineering
Job Description: GCP Data Engineer
Data Engineer (GCP Data Migration) is responsible for designing, developing, and executing large-scale data migration and modernization initiatives on the Google Cloud Platform (GCP). The role focuses on migrating data from legacy databases, on-premises systems, and other cloud platforms to GCP services such as BigQuery, Cloud Storage, Dataproc, Dataflow, Dataplex, and Cloud SQL. The Data Engineer ensures secure, scalable, high-performance data pipelines while maintaining data quality, governance, and compliance standards.
The role collaborates closely with Solution Architects, Application Teams, Business Stakeholders, Data Analysts, and Cloud Engineers to enable successful cloud transformation programs.
Key Responsibilities
- Data Migration & Modernization:
- Assess existing data platforms and define migration strategies to GCP.
- Design and execute large-scale database and data warehouse migrations.
- Develop source-to-target mappings, transformation rules, and migration plans.
- Migrate structured, semi-structured, and unstructured data to GCP platforms.
- Perform data validation, reconciliation, and quality assurance activities.
- Support migration cutover planning, rollback procedures, and production deployments.
- Data Engineering & Pipeline Development:
- Build scalable ETL/ELT pipelines using GCP-native services.
- Design batch and real-time data processing solutions.
- Develop automated ingestion frameworks from multiple source systems.
- Optimize data pipelines for performance, scalability, and cost efficiency.
- Implement reusable migration and transformation frameworks.
- GCP Data Platform Implementation:
- Build and manage solutions using:
- BigQuery
- Cloud Storage
- Dataflow
- Dataproc
- Pub/Sub
- Cloud Composer
- Dataplex
- Cloud SQL
- AlloyDB
- BigLake
- Support enterprise data lake, lakehouse, and warehouse implementations.
- Configure monitoring, logging, and performance optimization mechanisms.
- Data Quality & Governance:
- Implement data quality checks and reconciliation frameworks.
- Ensure adherence to data governance and security standards.
- Manage metadata, lineage, and cataloging using Dataplex and Data Catalog.
- Implement data encryption, masking, and access controls.
- Automation & DevOps:
- Implement CI/CD pipelines for data workloads.
- Automate deployment and infrastructure provisioning.
- Support Infrastructure as Code (IaC) implementations.
- Establish monitoring and operational support processes.
- Stakeholder Collaboration:
- Participate in migration workshops and architecture discussions.
- Estimate migration effort and identify risks and dependencies.
- Provide technical guidance and best practices.
- Support UAT, production migration, and hypercare activities.
Required Skills
- GCP Data Engineering Services:
- BigQuery
- Cloud Storage
- Dataflow
- Dataproc
- Cloud Composer (Airflow)
- Pub/Sub
- Dataplex
- BigLake
- AlloyDB
- Cloud SQL
- Data Fusion
- Datastream
- Data Migration Expertise:
- Legacy to Cloud migrations
- Data Warehouse migrations
- Database migrations
- ETL to ELT modernization
- Cross-cloud migrations
- Data validation and reconciliation
- Data profiling and cleansing
- Programming & Development:
- Python
- SQL
- PySpark
- Java (preferred)
- Scala (preferred)
- Shell Scripting
- Database Technologies:
- Oracle
- SQL Server
- PostgreSQL
- MySQL
- MongoDB
- Cassandra
- Teradata
- Snowflake
- Big Data Technologies
- Apache Spark
- Hadoop Ecosystem
- Kafka
- Hive
- Delta Lake
- DevOps & Automation:
- Git
- GitHub
- GitLab
- Jenkins
- Terraform
- CI/CD Pipelines
- Docker
- Kubernetes (preferred)
Required Qualifications
- working with GCP Data Services.
- Experience in enterprise-scale migration and modernization projects.
- Hands-on experience with BigQuery and Dataflow implementations.