Senior Data Engineer

Source Meridian; Colombia; Presencial; COP 90.000.000 - 120.000.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Permanent contract

High-impact healthcare projects

Learning and growth opportunities

Descripción de la vacante

Source Meridian is seeking a Senior Data Engineer to join our team and help scale a Databricks-based data platform for a US-based client. You will design and implement ingestion pipelines from diverse sources, optimize Spark workloads, and structure data in Delta Lake following bronze/silver/gold patterns.

You will collaborate with the DBA team on governance and data quality. The role requires 5+ years in data engineering, strong Databricks production experience, and an autonomous, proactive

Formación

• 5+ years of experience as a Data Engineer.
• Proven Databricks production experience with Delta Lake and Unity Catalog.
• Advanced Python for data engineering.
• Strong Spark performance tuning and debugging skills.
• AWS data platform fundamentals (S3, IAM, networking).
• Solid SQL skills and experience designing ingestion pipelines.
• Senior-level autonomy and ability to work with ambiguity.
• English proficiency at B2+ or higher.

Responsabilidades

• Design, build, and maintain data ingestion pipelines into Databricks from diverse sources.
• Develop and optimize Spark workloads (batch and streaming).
• Model data in Delta Lake following medallion-style (bronze/silver/gold).
• Collaborate with DBA and Data & Analytics teams on governance and data quality.
• Monitor, troubleshoot, and optimize pipelines for reliability and cost.
• Document pipelines and design decisions for maintainability.

Conocimientos

Databricks

Python

Apache Spark

Delta Lake

SQL

AWS

Data ingestion

English (B2+)

Herramientas

Terraform

Airflow

dbt

CI/CD

Kubernetes

Unity Catalog

Descripción del empleo

We’re looking for a Senior Data Engineer to join Source Meridian.

About Source Meridian

Source Meridian is a development software company that works to solve the industry’s most challenging problems in healthcare practices. We are laser focused on specific technologies in the healthcare and life science industries: Healthcare technology, artificial intelligence, and healthcare interoperability.

About The Role

We're growing a data engineering team for a US-based client whose entire data platform runs on Databricks on AWS. The immediate priority is building out and scaling their data ingestion layer — bringing in data from a wide range of source systems reliably, efficiently, and in a way that holds up as the platform grows. You'll work closely with two client-side teams: The DBA team, who own data governance and are the authority on source systems, access, and data stewardship.

What You'll Do

• Design, build, and maintain ingestion pipelines into Databricks from a variety of source systems.
• Build and optimize Spark workloads in Python — batch and streaming.
• Model and structure data in Delta Lake following medallion-style patterns (bronze / silver / gold).
• Partner with the DBA team on governance, access, and data quality expectations, and with the Data & Analytics team so ingested data actually serves their reporting and strategy needs.
• Monitor, troubleshoot, and tune pipelines for reliability, cost, and performance.
• Document pipelines, data contracts, and design decisions so the work is maintainable by the team.

Required Qualifications

• 5+ years of experience as a Data Engineer
• Strong, demonstrable Databricks experience. Someone who knows the platform well in production, not just conceptually. Delta Lake, workflows/jobs, clusters and compute configuration, Unity Catalog.
• Advanced Python for data engineering.
• Strong Apache Spark — including an understanding of how to diagnose and fix performance problems, not just write working code.
• AWS fundamentals as they relate to a data platform (S3, IAM, networking basics, secrets management).
• SQL at level.
• Experience designing ingestion pipelines from multiple heterogeneous source systems.
• Senior-level autonomy: able to take an ambiguous requirement, ask the right questions, and deliver.
• B2+ or superior english level

Nice to have

• Experience connecting Databricks to on-premises systems (networking, private connectivity, gateways, CDC from on-prem databases).
• Experience with other cloud data platforms — Snowflake, BigQuery, Synapse, Redshift.
• Streaming and CDC tooling (Structured Streaming, Auto Loader, Debezium, Kafka).
• Infrastructure as code (Terraform, Databricks Asset Bundles) and CI/CD for data pipelines.
• Orchestration beyond Databricks Workflows (Airflow, dbt).
• Databricks certifications (Data EngineerAssociate / Professional).
• Prior experience working with governance-heavy or regulated data environments.

What We Offer

• Permanent contract
• High-impact projects in the healthcare industry
• Learning and continuous growth environment
• Competitive salary based on experience
• Collaborative and international team

Apply only if you reside in Colombia

At Source Meridian, you’ll be part of a high-impact tech-health company, building products that truly make a difference.

We’d love to meet you