Senior Data Engineer - Remote (LATAM) | Databricks & DataOps

Insight Global; San Gil; Remote; COP 180,000,000 - 240,000,000; Full time

Job summary

Insight Global is seeking a senior data engineer to design, build, and maintain production-grade batch and streaming pipelines using Databricks, Lakeflow Declarative Pipelines, and dbt. You will collaborate with cross-functional teams to deliver analytics-ready datasets and data models, review designs, and mentor junior engineers.

Remote role open in Brazil or Colombia; you will participate in sprint rituals, drive data governance with Unity Catalog and Immuta, and advance DataOps, CI/CD, and

Qualifications

• 7+ years of data engineering and architecture experience.
• Deep, hands-on production experience with Databricks and lakehouse platforms.
• Strong Spark experience, including PySpark and Spark SQL.
• Experience building and operating scalable batch and streaming pipelines.
• Experience with declarative pipeline frameworks, including Lakeflow Declarative Pipelines.
• Strong experience with dbt, Databricks Workflows, and Apache Airflow for data transformation and orchestration.
• Advanced SQL and strong Python programming skills.
• Solid understanding of lakehouse architecture, medallion/layered data modeling, and data warehousing.
• Production experience with Databricks Unity Catalog.
• Cloud Experience (AWS, Azure, GCP) and data cataloging tools.

Responsibilities

• Design, build, and maintain production-grade batch and streaming pipelines.
• Monitor and troubleshoot pipeline reliability, performance, and cost.
• Implement centralized data governance and access controls with Unity Catalog and Immuta.
• Build DataOps standards, CI/CD automation, and automated data-quality controls.
• Partner with cross-functional stakeholders to deliver curated, analytics-ready datasets and data models.
• Review pipeline designs and code while guiding mid-level and junior engineers.
• Evaluate emerging Databricks capabilities and recommend platform improvements.

Skills

Databricks

Spark / PySpark

SQL

Python

Airflow

CI/CD

Data governance

Data modeling

DataOps

Lakehouse

Tools

Lakeflow Declarative Pipelines

dbt

Unity Catalog

Immuta

Databricks Workflows

Apache Airflow

Confluent Kafka

Fivetran

Alation

Databricks Serverless compute

Job description

Insight Global is seeking a senior data engineer to design, build, and maintain production-grade batch and streaming pipelines using Databricks, Lakeflow Declarative Pipelines, and dbt. You will collaborate with cross-functional teams to deliver analytics-ready datasets and data models, review designs, and mentor junior engineers.

Remote role open in Brazil or Colombia; you will participate in sprint rituals, drive data governance with Unity Catalog and Immuta, and advance DataOps, CI/CD, and