177. Senior Data Engineer

Source Meridian; Medellín; Presencial; COP 100.000.000 - 160.000.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Permanent contract

High-impact healthcare projects

Continuous learning & growth

Competitive salary based on experience

Collaborative, international team

Descripción de la vacante

Source Meridian seeks a Senior Data Engineer to grow a data team supporting a US-based client. You will design and scale ingestion pipelines into Databricks on AWS, optimize Spark workloads in Python, and implement Delta Lake medallion patterns (bronze/silver/gold).

Collaborate with the DBA team on governance and data quality while ensuring ingestion serves reporting and analytics needs. Strong autonomy required; English B2+ preferred.

Formación

• 5+ years of experience as a Data Engineer.
• Strong, demonstrable Databricks experience in production.
• Advanced Python for data engineering.
• Strong Apache Spark with performance debugging.
• AWS fundamentals for data platforms (S3, IAM, networking, secrets).
• SQL at a strong level.
• Experience designing ingestion pipelines from multiple heterogeneous source systems.
• Senior-level autonomy to handle ambiguous requirements.
• B2+ or superior English level.

Responsabilidades

• Design, build, and maintain ingestion pipelines into Databricks from diverse sources.
• Build and optimize Spark workloads in Python — batch and streaming.
• Model and structure data in Delta Lake following medallion-style patterns (bronze/silver/gold).
• Partner with the DBA team on governance, access, and data quality, and with Data & Analytics to serve reporting needs.
• Monitor, troubleshoot, and tune pipelines for reliability, cost, and performance.
• Document pipelines, data contracts, and design decisions for maintainability.

Conocimientos

Databricks

Python

Apache Spark

SQL

Data ingestion design

English proficiency

Data governance

Cloud security basics

Herramientas

Delta Lake

Terraform

Airflow

dbt

Kafka

Snowflake

BigQuery

Redshift

Descripción del empleo

We’re looking for a Senior Data Engineer to join Source Meridian.

About Source Meridian

Source Meridian is a development software company that works to solve the industry’s most challenging problems in healthcare practices. We are laser focused on specific technologies in the healthcare and life science industries: Healthcare technology, artificial intelligence, and healthcare interoperability.

About the role

We're growing a data engineering team for a US-based client whose entire data platform runs on Databricks on AWS. The immediate priority is building out and scaling their data ingestion layer — bringing in data from a wide range of source systems reliably, efficiently, and in a way that holds up as the platform grows. You'll work closely with two client-side teams: The DBA team, who own data governance and are the authority on source systems, access, and data stewardship.

What You'll do

• Design, build, and maintain ingestion pipelines into Databricks from a variety of source systems.
• Build and optimize Spark workloads in Python — batch and streaming.
• Model and structure data in Delta Lake following medallion-style patterns (bronze / silver / gold).
• Partner with the DBA team on governance, access, and data quality expectations, and with the Data &Analytics team so ingested data actually serves their reporting and strategy needs.
• Monitor, troubleshoot, and tune pipelines for reliability, cost, and performance.
• Document pipelines, data contracts, and design decisions so the work is maintainable by the team.

Required Qualifications

5+ years of experience as a Data Engineer

Strong, demonstrable Databricks experience. Someone who knows the platform well in production, not just conceptually. Delta Lake, workflows/jobs, clusters and compute
configuration, Unity Catalog.

Advanced Python for data engineering.

Strong Apache Spark — including an understanding of how to diagnose and fix performance problems, not just write working code.

AWS fundamentals as they relate to a data platform (S3, IAM, networking basics, secrets management).

SQL at a strong level.

Experience designing ingestion pipelines from multiple heterogeneous source systems.

Senior-level autonomy: able to take an ambiguous requirement, ask the right questions, and deliver.

• B2+ or superior english level

Nice to have

• Experience connecting Databricks to on-premises systems (networking, private connectivity, gateways, CDC from on-prem databases).
• Experience with other cloud data platforms — Snowflake, BigQuery, Synapse, Redshift.
• Streaming and CDC tooling (Structured Streaming, Auto Loader, Debezium, Kafka).
• Infrastructure as code (Terraform, Databricks Asset Bundles) and CI/CD for data pipelines.
• Orchestration beyond Databricks Workflows (Airflow, dbt).
• Databricks certifications (Data EngineerAssociate / Professional).
• Prior experience working with governance-heavy or regulated data environments.

What We Offer

• Permanent contract
• High-impact projects in the healthcare industry
• Learning and continuous growth environment
• Competitive salary based on experience
• Collaborative and international team

Apply only if you reside in Colombia

AtSource Meridian, you’ll be part of a high-impacttech-healthcompany, building products that truly make a difference.