Remote Senior Data Engineer: Databricks, Delta Lake, Python

Encora10; Colombia; Remote; COP 286,469,000 - 413,789,000; Full time

Job summary

Coforge is seeking a Senior Data Engineer (Earnix) to design and optimize enterprise-grade data pipelines on Azure Databricks and Delta Lake, with strong governance and cross-functional collaboration across pricing, actuarial, underwriting, Data Science, BI, and QA teams.

The role requires 7+ years in data engineering with hands-on Databricks, Python/PySpark, SQL, dbt, and CI/CD experience, plus expertise in Unity Catalog and cloud governance. This is a remote LATAM position.

Qualifications

• 7+ years of Data Engineering experience, including strong recent experience with enterprise Databricks platforms.
• Strong production experience with Azure Databricks, Spark, Delta Lake, Workflows/Jobs, Databricks SQL, and platform optimization.
• Advanced hands-on experience with dbt models, macros, tests, documentation, incremental patterns, and CI/CD integration.
• Strong Python and PySpark skills for transformations, reusable frameworks, integrations, validation, and automated testing.
• Advanced SQL skills, including query optimization, data reconciliation, and analytical data modeling.
• Strong experience with enterprise ELT/ETL, batch pipelines, incremental processing, data quality, and production support.
• Practical experience with Unity Catalog or comparable data governance, lineage, and access-control capabilities.
• Experience with CI/CD, Git, automated deployments, peer reviews, and environment promotion for data products.
• Strong Azure experience, including cloud storage, identity, security, and integration patterns.
• Proven ownership of production data pipelines from design through deployment, monitoring, and incident resolution.

Responsibilities

• Design, develop, and optimize production-grade data pipelines and governed data products using Azure Databricks, dbt, Python/PySpark, SQL, and Delta Lake.
• Build conformed Silver and Gold data models for pricing-related data and model inputs.
• Implement scalable Lakehouse engineering practices with governance, quality, and observability.
• Collaborate with pricing, actuarial, underwriting, Data Science, BI, Architecture, and QA teams while providing technical leadership.

Skills

Azure Databricks

dbt

Python

PySpark

SQL

Delta Lake

Unity Catalog

Azure

CI/CD

Data Engineering

Earnix

Tools

Terraform

Power BI

Kafka

Job description

Coforge is seeking a Senior Data Engineer (Earnix) to design and optimize enterprise-grade data pipelines on Azure Databricks and Delta Lake, with strong governance and cross-functional collaboration across pricing, actuarial, underwriting, Data Science, BI, and QA teams.

The role requires 7+ years in data engineering with hands-on Databricks, Python/PySpark, SQL, dbt, and CI/CD experience, plus expertise in Unity Catalog and cloud governance. This is a remote LATAM position.