Senior Data Engineer — AWS Spark for Healthcare Data

sourcemeridian; Medellín; In loco; COP 191.528.000 - 383.056.000; Tempo pieno

Vantaggi offerti da questo lavoro

Health and well-being benefits

Learning and growth environment

Descrizione del lavoro

Source Meridian is seeking a Senior Data Engineer to build an AWS-native data platform processing healthcare claims data using Spark on S3 with Parquet storage. You will implement tokenization workflows and ensure data integrity while collaborating with US stakeholders.

You will design, test, and operate ETL/ELT processes, optimize performance, and contribute to dbt models. This role requires strong English and healthcare data experience, and is based in Colombia.

Competenze

• 5+ years of professional experience in Data Engineering.
• Strong experience with Apache Spark (PySpark or Scala).
• Hands-on AWS data stack experience with S3/Parquet and Athena.
• Familiarity with Airflow and dbt for ETL/ELT pipelines.
• Excellent SQL skills and data modeling fundamentals.
• Advanced English; able to lead technical discussions with US stakeholders.

Mansioni

• Build and maintain Spark pipelines to process large Parquet datasets on S3.
• Implement tokenization workflows and dataset intersections/joins.
• Process healthcare claims data with identity mapping and data integrity.
• Orchestrate pipelines with Airflow or AWS-native tools; ensure reliability.
• Develop testable ETL/ELT processes with monitoring and retries.
• Optimize Spark, S3 layout, and Athena query patterns for cost/performance.

Conoscenze

Spark

PySpark/Scala

SQL

Leadership

Communication

Data modeling

Strumenti

AWS Data Stack

S3 Parquet

Athena

Airflow

dbt

Descrizione del lavoro

Source Meridian is seeking a Senior Data Engineer to build an AWS-native data platform processing healthcare claims data using Spark on S3 with Parquet storage. You will implement tokenization workflows and ensure data integrity while collaborating with US stakeholders.

You will design, test, and operate ETL/ELT processes, optimize performance, and contribute to dbt models. This role requires strong English and healthcare data experience, and is based in Colombia.