GCP Data Engineer

Whitehall Resources Ltd; City Of London; On-site; GBP 60,000 - 100,000; Full time

Job summary

An established industry player is seeking a Senior GCP Data Engineer to join their Cyber Security domain. This role offers an exciting opportunity to leverage your extensive data engineering experience and cloud technology skills to deliver exceptional data capabilities. You will be responsible for developing robust data pipelines, optimizing performance, and collaborating with product managers to drive business transformation. If you are passionate about cloud solutions and eager to make an impact in a dynamic environment, this position is perfect for you.

Qualifications

• 8+ years of experience in data engineering with GCP focus.
• Proficiency in GCP services and Python for data solutions.

Responsibilities

• Design and maintain scalable backend systems and APIs.
• Develop data pipelines for Google Cloud environments.

Skills

Google Cloud Platform (GCP)

BigQuery

Python

SQL

Data Engineering

Terraform

CI/CD Pipelines

Data Lakes

Data Transformation

Data Integration

Education

Bachelor's degree in Computer Science or related field

Tools

Cloud Composer

DataFlow

Pub/Sub

Cloud SQL

Compute Engine

GKE

Job description

Whitehall Resources requires a GCP Data Engineer to work with a key client on an initial 12-month contract.

*This role involves on-site work 3 days per week.

*Inside IR35.

The Role

An excellent opportunity has emerged for a Senior GCP Data Engineer to join the Cyber Security domain. In this role, you will be responsible for providing top-tier data capabilities, leveraging your engineering skills, and embracing the possibilities offered by cloud technology.

Key Responsibilities

1. API Development: Design, develop, and maintain robust and scalable backend systems and APIs.
2. Data Ingestion: Develop and maintain data pipelines to extract data from various sources and load it into Google Cloud environments.
3. Data Transformation: Implement data transformation processes, including data cleansing, normalization, and aggregation, to ensure data quality and consistency.
4. Data Modeling: Develop and maintain data models and schemas to support efficient data storage and retrieval in Google Cloud platforms.
5. Data Integration: Integrate data from multiple sources, both on-premises and cloud-based, using Cloud Composer or other relevant tools.
6. Data Lakes: Build data lakes using Google Cloud services such as BigQuery.
7. Performance Optimization: Optimize data pipelines and queries for improved performance and scalability in Google Cloud environments.
8. Collaboration: Work with product managers to ensure superior product delivery to drive business value & transformation.
9. Documentation: Document data engineering processes, data flows, and system configurations for future reference and knowledge sharing.

Your Profile

Essential skills/knowledge/experience:

• 8+ years of experience in data engineering, with a strong focus on Google Cloud Platform (GCP)-based solutions.
• Proficiency in GCP services such as BigQuery, DataProc, Cloud SQL, DataFlow, Pub/Sub, Cloud Data Fusion, Cloud Composer, Python, SQL.
• Experience designing, developing, and deploying scalable, reliable, and secure cloud solutions using GCP.
• Ability to translate business requirements into technical specifications.
• Proficiency with core GCP services like Compute Engine, GKE, Cloud Storage, Cloud Functions, Cloud SQL, and BigQuery.
• Experience managing GCP networking configurations.
• Using IaC tools like Terraform for infrastructure automation.
• Implementing and managing CI/CD pipelines and automating deployments.
• Applying security best practices to protect cloud infrastructure and data.
• Identifying and resolving performance bottlenecks.

Desirable skills/knowledge/experience:

• Experience with data analytics and Big Data technologies.
• Knowledge of cloud security standards and compliance.
• Experience with agile development methodologies.
• GCP certifications (e.g., Google Cloud Certified Professional Cloud Developer) are a plus.

All applicants must be eligible to work in the specified country/location.

Whitehall Resources is an equal opportunity employer valuing diversity and inclusion. All qualified applicants will receive consideration without regard to race, religion, gender identity, sexual orientation, or other characteristics.