Senior Data Engineer | AWS & GCP

Reach IT; Portugal; On-site; EUR 60,000 - 90,000; Full time

Job summary

Reach IT is seeking a Senior Data Engineer to join a Life Sciences data engineering project supporting a pharmaceutical organization focused on research, development and manufacturing of innovative therapies.

The role centers on building and evolving an enterprise cloud data platform that integrates research, laboratory, manufacturing and operational datasets to power analytics, data quality and scientific decision-making, using AWS, GCP, Python, SQL and PySpark.

Qualifications

• Senior Data Engineer with AWS and GCP experience.
• Experience building enterprise cloud data platforms.
• Background in data ingestion, processing and analytics.

Responsibilities

• Develop AWS ingestion and processing pipelines using S3, Glue and EMR.
• Implement GCP analytics pipelines with Cloud Storage, Dataflow and BigQuery.
• Create Python, SQL, and PySpark transformations to standardize data.
• Design ETL workflows and manage schema evolution and loads.
• Configure AWS Step Functions and Airflow for pipeline orchestration.
• Improve Spark and BigQuery performance, reduce costs.
• Enforce data quality checks and governance with Glue Data Quality and Dataplex.
• Automate cloud infrastructure and data pipeline deployments with Terraform and GitHubActions.
• Collaborate with Data Architects, Analytics Engineers, and pharma teams.

Skills

ETL pipeline design

Data governance awareness

Performance optimization

Tools

AWS

GCP

Python

SQL

PySpark

ETL pipelines

Data Lake & Data Warehouse architectures

Apache Airflow

Terraform

Git & GitHub Actions

Job description

Our nearshore expertise sets us apart. We specialize in connecting experienced technology professionals with complex international engineering environments, supporting organizations in building scalable, reliable, and high-performance data platforms.

We are looking for a Senior Data Engineer | AWS & GCP

To join a Life Sciences Data Engineering project, supporting a pharmaceutical organization specialized in research, development, and manufacturing of innovative therapies.

The project focuses on the evolution of an enterprise cloud data platform, integrating research, laboratory, manufacturing, and operational datasets to support analytics, data quality, and scientific decision-making.

Skills

• AWS: S3, Glue, EMR and Step Functions for ingestion, processing and orchestration;
• GCP: BigQuery, Cloud Storage, Dataflow and Cloud Composer for analytical/DWH;
• Python, SQL and PySpark for data transformations and distributed processing;
• ETL pipelines for enterprise data integration and processing;
• Data Lake & Data Warehouse architectures for scalable storage and analytical consumption;
• Apache Airflow for pipeline orchestration and dependency management;
• Data Quality & Governance for validation, lineage and traceability;
• Terraform, Git & GitHub Actions for infrastructure automation and controlled deployments;
• Performance Optimization for distributed workloads and analytical queries.

Responsibilities

• Develop AWS-based ingestion and processing pipelines, using S3 to organize source datasets, Glue for extraction and transformation, and EMR for large-scale PySpark processing;
• Implement GCP analytical data pipelines, using Cloud Storage for staging, Dataflow for transformation workflows, and BigQuery for curated analytical datasets and reporting consumption;
• Build Python, SQL, and PySpark transformations to standardize, cleanse, validate, and combine data from research, laboratory, manufacturing, and enterprise systems;
• Design and maintain ETL workflows connecting data processing and analytical layers, managing schema evolution, incremental loads, data dependencies, and processing efficiency;
• Configure AWS Step Functions and Cloud Composer / Apache Airflow to coordinate pipeline execution, scheduling, monitoring, retries, and recovery mechanisms;
• Optimize Spark processing jobs and BigQuery workloads, improving partitioning strategies, query execution, storage efficiency, and cloud processing costs;
• Implement data quality checks and validation rules using AWS Glue Data Quality, PySpark and SQL, leveraging Google Cloud Dataplex for data quality monitoring, metadata management and lineage;
• Automate cloud infrastructure and data pipeline deployments through Terraform and GitHubActions;
• Collaborate with Data Architects, Analytics Engineers, and pharmaceutical domain teams to deliver reusable data models, reliable transformation frameworks, and scalable data products.