Senior Data Engineer: Databricks & AWS Ingestion

Source Meridian; Medellín; Presencial; COP 100.000.000 - 160.000.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Permanent contract

High-impact healthcare projects

Continuous learning & growth

Competitive salary based on experience

Collaborative, international team

Descripción de la vacante

Source Meridian seeks a Senior Data Engineer to grow a data team supporting a US-based client. You will design and scale ingestion pipelines into Databricks on AWS, optimize Spark workloads in Python, and implement Delta Lake medallion patterns (bronze/silver/gold).

Collaborate with the DBA team on governance and data quality while ensuring ingestion serves reporting and analytics needs. Strong autonomy required; English B2+ preferred.

Formación

• 5+ years of experience as a Data Engineer.
• Strong, demonstrable Databricks experience in production.
• Advanced Python for data engineering.
• Strong Apache Spark with performance debugging.
• AWS fundamentals for data platforms (S3, IAM, networking, secrets).
• SQL at a strong level.
• Experience designing ingestion pipelines from multiple heterogeneous source systems.
• Senior-level autonomy to handle ambiguous requirements.
• B2+ or superior English level.

Responsabilidades

• Design, build, and maintain ingestion pipelines into Databricks from diverse sources.
• Build and optimize Spark workloads in Python — batch and streaming.
• Model and structure data in Delta Lake following medallion-style patterns (bronze/silver/gold).
• Partner with the DBA team on governance, access, and data quality, and with Data & Analytics to serve reporting needs.
• Monitor, troubleshoot, and tune pipelines for reliability, cost, and performance.
• Document pipelines, data contracts, and design decisions for maintainability.

Conocimientos

Databricks

Python

Apache Spark

SQL

Data ingestion design

English proficiency

Data governance

Cloud security basics

Herramientas

Delta Lake

Terraform

Airflow

dbt

Kafka

Snowflake

BigQuery

Redshift

Descripción del empleo

Source Meridian seeks a Senior Data Engineer to grow a data team supporting a US-based client. You will design and scale ingestion pipelines into Databricks on AWS, optimize Spark workloads in Python, and implement Delta Lake medallion patterns (bronze/silver/gold).

Collaborate with the DBA team on governance and data quality while ensuring ingestion serves reporting and analytics needs. Strong autonomy required; English B2+ preferred.