Médico General – Consulta Externa, Jornada Tarde (San Gil)

Fundacion Avanzar Fos; Santander; Remote; COP 120,000,000 - 180,000,000; Part time

Job summary

Insight Global is seeking a senior data engineer to design, build, and maintain production-grade batch and streaming pipelines using Databricks, Lakeflow Declarative Pipelines, and dbt. You will collaborate in sprint ceremonies and mentor junior engineers while driving data governance and CI/CD automation.

This is a 3-month contract with potential yearly extensions and remote work options in Brazil or Colombia.

Qualifications

• 7+ years of data engineering and architecture experience.
• Production experience with Databricks and lakehouse platforms.
• Strong Spark experience including PySpark and Spark SQL.
• Experience with batch and streaming pipelines.
• Experience with Lakeflow Declarative Pipelines.
• Strong Python programming and SQL skills.
• Cloud experience (AWS, Azure, GCP).
• Databricks Unity Catalog experience.
• Familiarity with Confluent Kafka and Fivetran.
• Immuta and data catalog tooling experience.

Responsibilities

• Design, build, and maintain production-grade batch and streaming pipelines.
• Partner with cross-functional stakeholders to deliver curated analytics-ready datasets and data models.
• Review pipeline designs and guide mid-level and junior engineers.
• Evaluate Databricks capabilities and recommend platform improvements.

Skills

Data engineering

Spark

SQL

Python

Databricks

Airflow

Cloud

Databricks Unity Catalog

Data modeling

Scala/Java

dbt

Fivetran

Immuta

Alation

Tools

Databricks

Lakeflow Declarative Pipelines

dbt

Apache Airflow

Fivetran

Immuta

Alation

Unity Catalog

Kafka

Job description

Job Description

Insight Global is looking for a senior data engineer to design, build, and maintain production-grade batch and streaming pipelines using Databricks, Lakeflow Declarative Pipelines, and dbt for our client. This person will participate in sprint planning, standups, retrospectives, and backlog grooming. You will monitor and troubleshoot pipeline reliability, performance, and cost, implement centralized data governance and access controls with Unity Catalog and tools such as Immuta, and build DataOps standards, CI/CD automation, and automated data-quality controls. You will be responsible for partnering with cross-functional stakeholders to deliver curated, analytics-ready datasets and data models, and review pipeline designs and code while guiding mid-level and junior engineers. Lastly, you will evaluate emerging Databricks capabilities, including Lakeflow, AI/BI, and SQL AI Functions and recommend improvements to platform architecture and engineering practices. This is a 3 month contract with yearly anticipated extensions remote in Brazil or Colombia.

Skills and Requirements

• 7+ years of data engineering and architecture experience
• Deep, hands-on production experience with Databricks and modern lakehouse platforms
• Strong Apache Spark experience, including PySpark and Spark SQL
• Experience building and operating scalable batch and streaming pipelines
• Experience with declarative pipeline frameworks, including Lakeflow Declarative Pipelines
• Strong experience with dbt, Databricks Workflows, and Apache Airflow for data transformation and orchestration
• Advanced SQL and strong Python programming skills
• Solid understanding of lakehouse architecture, medallion/layered data modeling, and data warehousing
• Production experience with Databricks Unity Catalog
• Cloud Experience (AWS, Azure, GCP)
• Scala or Java experience
• Databricks Certified Data Engineer Professional certification
• Confluent Kafka and Fivetran experience
• Immuta experience alongside Unity Catalog
• Alation or similar data catalog and lineage tooling
• Declarative Automation Bundles
• Photon, Liquid Clustering, and Databricks Serverless compute

We are a company committed to creating diverse and inclusive environments where people can bring their full, authentic selves to work every day. We are an equal employment opportunity/affirmative action employer that believes everyone matters. Qualified candidates will receive consideration for employment without regard to race, color, ethnicity, religion, sex (including pregnancy), sexual orientation, gender identity and expression, marital status, national origin, ancestry, genetic factors, age, disability, protected veteran status, military or uniformed service member status, or any other status or characteristic protected by applicable laws, regulations, and ordinances. If you need assistance and/or a reasonable accommodation due to a disability during the application or the recruiting process,