Data Engineer (M/F/D)

T-Systems; Madrid; Hybrid; EUR 30,000 - 42,000; Full time

Benefits offered by this job

Hybrid work model

Health insurance

Meal vouchers

Telemedicine

Life and accident insurance

26+ vacation days

Job summary

T-Systems in Madrid, Spain, seeks a Junior Data Engineer to join the DBIZ team. You will work on data ingest, development of data products on ODE, and operations of data products across environments, leveraging GCP, Hive, PySpark, and Airflow.

You will deploy infrastructure, implement CI/CD pipelines, and build Python applications for cloud services, contributing to secure, scalable data solutions in a hybrid work setup.

Qualifications

• Experience with Google Cloud Platform (GCP)
• Familiarity with Terraform for infrastructure
• Proficiency in Python for application development
• Experience with GitLab CI/CD automation
• Knowledge of network architectures and security configurations
• Experience managing core GCP services
• Knowledge of Hive and PySpark for data processing
• Experience with Airflow for orchestration
• Experience integrating diverse data sources

Responsibilities

• Deploy and manage cloud infrastructure on GCP across DEV/TEST/PROD
• Build data ingestion and transformation pipelines using Hive, PySpark and Airflow
• Develop and run Python applications for GCP services
• Implement CI/CD pipelines with GitLab Magenta
• Manage security policies, IAM roles, and Secrets in Secret Manager
• Integrate data from CDI, FTP servers, TARDIS APIs and GCS
• Work with GKE, Cloud Run, BigQuery, Compute Engine, and Composer via Terraform
• Support AI solutions using Vertex AI

Skills

GCP familiarity

Terraform

Python

GitLab CI/CD

Network security

Core GCP services

Hive

PySpark

Airflow

Data sources integration

Tools

Terraform

GKE / Cloud Run / BigQuery

Job description

Job Description We are looking for a skilled Junior Data Engineer to join our team in DBIZ. The Technik Value Stream teams are responsible for Data Ingest on ODE, development of the relevant data products on ODE, and Operations of the data products on ODE.

Your Task

We are looking for a skilled Junior Data Engineer to join our team in DBIZ. The Technik Value Stream teams are responsible for Data Ingest on ODE, development of the relevant data products on ODE, and Operations of the data products on ODE.

Responsibilities

• Infrastructure Deployment & Management: Efficiently deploy and manage infrastructure on Google Cloud Platform (GCP) including network architectures (Shared VPC, Hub-and-Spoke), security implementations (IAM, Secret Manager, firewalls, Identity-Aware Proxy), DNS configuration, VPN, and Load Balancing.
• Data Processing & Transformation: Utilize Hadoop cluster with Hive for querying data, and PySpark for data transformations. Implement job orchestration using Airflow.
• Core GCP Services Management: Work extensively with services like Google Kubernetes Engine (GKE), Cloud Run, BigQuery, Compute Engine, and Composer, all managed through Terraform.
• Application Implementation: Develop and implement Python applications for various GCP services.
• CI/CD Pipelines: Integrate and manage GitLab Magenta CI/CD pipelines for automating cloud deployment, testing, and configuration of diverse data pipelines.
• Security & Compliance: Implement comprehensive security measures, manage IAM policies, secrets using Secret Manager, and enforce identity-aware policies.
• Data Integration: Handle integration of data sources from CDI, Datendrehscheibe (FTP servers), TARDIS API's and Google Cloud Storage (GCS).
• Multi-environment Deployment: Create and deploy workloads across Development (DEV), Testing (TEST), and Production (PROD) environments.
• AI Solutions: Implement AI solutions using Google's Vertex AI for building and deploying machine learning models.

Your Profile

We are looking for a skilled Junior Data Engineer with strong expertise in Google Cloud Platform infrastructure, data processing technologies, and automation tools to join our team in DBIZ.

Must Have Skills

• Familiarity with Google Cloud Platform (GCP)
• Familiarity with Terraform for infrastructure management
• Python programming for application implementation
• Familiarity with GitLab CI/CD for automation
• Familiarity with network architectures and security implementations
• Familiarity with management of core GCP services
• Knowledge in employing data processing tools like Hive and PySpark
• Knowledge in data orchestration tools like Airflow
• Familiarity with managing and integrating diverse data sources

Nice to Have Skills

• Certified GCP Cloud Architect
• Certified GCP Data Engineer

Qualifications

Skills Required

• Familiarity with GCP.
• Familiarity with Terraform for infrastructure management.
• Python for application implementation.
• Familiarity with GitLab CI/CD for automation.
• Familiarity with network architectures, security implementations, and management of core GCP services.
• Knowledge in employing data processing tools like Hive, PySpark, and data orchestration tools like Airflow.
• Familiarity with managing and integrating diverse data sources.
• Certified GCP Cloud Architect and Data Engineer would be beneficial.

Additional Information

What do we offer you?

Work environment & flexibility

• International, dynamic and collaborative environment.
• T-Social: social initiatives (sports, community, health, ...).
• Hybrid work model (remote/on-site).
• Flexible working hours.

Growth & development

• Customized training: access to Coursera to learn whatever you want, whenever you want.
• Weekly language classes (English & German).
• International Mentoring Sessions & Experience Days.

Compensation & benefits

• Flexible compensation plan (health insurance, meal vouchers, childcare, transport).
• Telemedicine.
• Life and accident insurance.
• Social fund.

Wellbeing & time off

• 26+ working days of vacation per year.
• Free access to specialist services (medical, legal, wellness).
• 100% salary coverage during medical leave.
• And many more advantages of being part of T-Systems!