GCP BigQuery Data Ingestion Architect (Hybrid)

Insight International (UK) Ltd; Leeds; Hybrid; GBP 60,000 - 80,000; Full time

Job summary

A leading cloud services firm is seeking a Cloud Data Loading Architect to design and implement automated data ingestion pipelines in Google Cloud Platform, focusing on BigQuery. The ideal candidate will have deep expertise in cloud data services, strong coding skills in SQL and Python, and a solid understanding of data governance. This role requires a blend of technical knowledge and strong communication skills to collaborate effectively with various teams and ensure efficient data processes. The position offers flexibility with a hybrid working model.

Qualifications

• Deep expertise in Google Cloud Data Services including BigQuery, Dataflow, and Pub/Sub.
• Hands-on experience designing data ingestion frameworks from various sources.
• Strong SQL and BigQuery optimization skills for efficient data processing.
• Proficient in Python or Java for data pipeline automation.
• Knowledge of cloud security and governance best practices.
• Experience with data quality frameworks and observability tools.
• Excellent communication skills for technical documentation and stakeholder engagement.

Responsibilities

• Design and implement ingestion pipelines for data loading into BigQuery.
• Define data loading frameworks and manage schema evolution.
• Create blueprints for governance and auditability of data.
• Establish data quality checks and validation rules.
• Optimize costs and access patterns in BigQuery.
• Collaborate with security teams on compliance and policies.
• Document technical processes and coach engineering teams.
• Troubleshoot data integration issues.

Skills

Google Cloud Data Services

Data Ingestion Engineering

SQL & BigQuery Optimization

Python

Cloud Security & Governance

Data Quality & Observability

Architectural Communication

Job description

A leading cloud services firm is seeking a Cloud Data Loading Architect to design and implement automated data ingestion pipelines in Google Cloud Platform, focusing on BigQuery. The ideal candidate will have deep expertise in cloud data services, strong coding skills in SQL and Python, and a solid understanding of data governance. This role requires a blend of technical knowledge and strong communication skills to collaborate effectively with various teams and ensure efficient data processes. The position offers flexibility with a hybrid working model.