Senior GCP Data Engineer

TN United Kingdom; London; On-site; GBP 60,000 - 100,000; Full time

Job summary

An excellent opportunity has emerged for a Senior GCP Data Engineer to join a leading investment bank in the Cyber Security domain. In this role, you will leverage your extensive experience in data engineering to provide top-tier data capabilities, utilizing the latest cloud technologies. You will be responsible for designing and maintaining scalable backend systems and APIs, developing data pipelines, and ensuring data quality through transformation processes. This innovative firm values collaboration and aims to drive business transformation through superior product delivery. If you're passionate about cloud technology and data engineering, this position offers a dynamic environment to enhance your skills and make a significant impact.

Qualifications

• 8+ years of experience in data engineering with a focus on GCP.
• Proficient in GCP services, including Data & AI services.
• Experience with CI/CD pipelines and cloud security best practices.

Responsibilities

• Design, develop, and maintain robust backend systems and APIs.
• Develop and maintain data pipelines for Google Cloud environments.
• Optimize data pipelines and queries for performance in GCP.

Skills

Data Engineering

Google Cloud Platform (GCP)

Python

SQL

Data Modeling

API Development

Data Ingestion

Data Transformation

Performance Optimization

Agile Development

Tools

Terraform

Cloud Composer

BigQuery

Cloud SQL

Kubernetes Engine (GKE)

Cloud Functions

Job description

Social network you want to login/join with:

An excellent opportunity has emerged for a Senior GCP Data Engineer to join the Cyber Security domain of an investment bank.

In this role, you will be responsible for providing top-tier data capabilities, leveraging your engineering skills, and embracing the possibilities offered by cloud technology.

Key responsibilities:

1. API Development: Design, develop, and maintain robust and scalable backend systems and APIs.
2. Data Ingestion: Develop and maintain data pipelines to extract data from various sources and load it into Google Cloud environments.
3. Data Transformation: Implement data transformation processes, including data cleansing, normalization, and aggregation, to ensure data quality and consistency.
4. Data Modeling: Develop and maintain data models and schemas to support efficient data storage and retrieval in Google Cloud platforms.
5. Data Integration: Integrate data from multiple sources, both on-premises and cloud-based, using Cloud Composer or other relevant tools.
6. Data Lakes: Build data lakes using Google Cloud services such as BigQuery.
7. Performance Optimization: Optimize data pipelines and queries for improved performance and scalability in Google Cloud environments.
8. Collaboration: Work with product managers to ensure superior product delivery to drive business value & transformation.
9. Documentation: Document data engineering processes, data flows, and system configurations for future reference and knowledge sharing.

Essential skills/knowledge/experience:

1. 8+ years of experience in data engineering, with a strong focus on Google Cloud Platform (GCP)-based solutions.
2. Proficiency in the GCP platform, particularly in Data & AI services (e.g., BigQuery, DataProc, Cloud SQL, DataFlow, Pub/Sub, Cloud Data Fusion, Cloud Composer, Python, SQL).
3. Designing, developing, and deploying scalable, reliable, and secure cloud-based solutions using GCP services.
4. Translating business requirements into technical specifications.
5. Proficiently utilizing core GCP services, such as Compute Engine, Kubernetes Engine (GKE), Cloud Storage, Cloud Functions, Cloud SQL, and BigQuery.
6. Implementing and managing GCP networking configurations.
7. Using IaC tools like Terraform to automate infrastructure provisioning and management.
8. Implementing and managing Continuous Integration/Continuous Delivery (CI/CD) pipelines.
9. Automating deployment processes. Implementing security best practices to protect cloud infrastructure and data.
10. Identifying and resolving performance bottlenecks.
11. Experience with data analytics and Big Data technologies.
12. Knowledge of cloud security best practices and compliance standards.
13. Experience with agile development methodologies.
14. GCP certifications (e.g., Google Cloud Certified Professional Cloud Developer) are a plus.