Data Engineer - AWS Spark & Healthcare Data

Source Meridian; Medellín; On-site; COP 60,000,000 - 120,000,000; Full time

Benefits offered by this job

Permanent contract

Learning and growth environment

Health and wellbeing benefits

Job summary

Source Meridian is seeking a Data Engineer to build and operate an AWS-native data platform handling healthcare claims data. You will design Spark pipelines and work with Parquet on S3, queries via Athena, and orchestrate with Airflow.

Join a privacy-conscious healthcare-focused team and contribute to dbt models when applicable. Ideal candidates have 2+ years in data engineering, strong Spark and AWS data stack experience, solid SQL, and excellent English to collaborate with US-based

Qualifications

• 2+ years of professional experience in Data Engineering.
• Strong experience with Apache Spark (PySpark/Scala), including joins, intersections, partitioning, and performance tuning.
• Strong hands-on experience with the AWS data stack, including S3 (Parquet datasets), Athena, Glue catalog, Lake Formation concepts.
• Experience building and operating pipelines using Airflow.
• Excellent SQL skills and solid data modeling fundamentals.
• Advanced English level for leading technical discussions with US-based stakeholders.

Responsibilities

• Build and maintain Spark pipelines to process large-scale Parquet datasets on S3.
• Process and deliver healthcare claims datasets for matched individuals, ensuring accurate identity mapping and data integrity.
• Orchestrate data pipelines using Airflow and/or AWS-native orchestration tools when appropriate.
• Develop reliable, testable, and observable ETL/ELT processes (retries, idempotency, monitoring, reprocessing).
• Optimize performance and cost across Spark jobs, S3 partitioning/layout, and Athena query patterns.
• Contribute to dbt models when applicable (transformations, documentation, data quality checks).
• Collaborate with cross-functional stakeholders in a healthcare environment, with a strong focus on privacy and secure data handling.

Skills

Spark

AWS

Airflow

SQL

English

Tools

dbt

S3

Athena

Glue

Job description

Source Meridian is seeking a Data Engineer to build and operate an AWS-native data platform handling healthcare claims data. You will design Spark pipelines and work with Parquet on S3, queries via Athena, and orchestrate with Airflow.

Join a privacy-conscious healthcare-focused team and contribute to dbt models when applicable. Ideal candidates have 2+ years in data engineering, strong Spark and AWS data stack experience, solid SQL, and excellent English to collaborate with US-based