Remote Senior Data Engineer: Azure Databricks & Delta Lake

Encora Inc.; Bogotá ciudad; À distance; COP 383 767 000 - 575 650 000; Plein temps

Résumé du poste

Coforge is hiring a Senior Data Engineer (Earnix) to design and implement production-grade data pipelines and governed data products using Azure Databricks, dbt, Python/PySpark, SQL, and Delta Lake. You will build Silver/Gold data models for pricing data and ensure scalable Lakehouse governance with Unity Catalog.

The role requires 7+ years of data engineering experience, strong Azure Databricks expertise, and ownership of end-to-end pipelines, with remote collaboration across pricing,

Qualifications

• 7+ years of Data Engineering experience, including strong recent experience with enterprise Databricks platforms.
• Strong production experience with Azure Databricks, Spark, Delta Lake, Workflows/Jobs, Databricks SQL, and platform optimization.
• Advanced hands-on experience with dbt models, macros, tests, documentation, incremental patterns, and CI/CD integration.
• Strong Python and PySpark skills for transformations, reusable frameworks, integrations, validation, and automated testing.
• Advanced SQL skills, including query optimization, data reconciliation, and analytical data modeling.
• Strong experience with enterprise ELT/ETL, batch pipelines, incremental processing, data quality, and production support.
• Practical experience with Unity Catalog or comparable data governance, lineage, and access-control capabilities.
• Experience with CI/CD, Git, automated deployments, peer reviews, and environment promotion for data products.
• Strong Azure experience, including cloud storage, identity, security, and integration patterns.
• Proven ownership of production data pipelines from design through deployment, monitoring, and incident resolution.

Responsabilités

• Design, develop, and optimize production-grade data pipelines and governed data products using Azure Databricks, dbt, Python/PySpark, SQL, and Delta Lake.
• Build conformed Silver and Gold data models for pricing and rating data.
• Implement scalable Lakehouse engineering practices with governance and performance optimization.
• Collaborate with pricing, actuarial, underwriting, Data Science, BI, Architecture, and Quality Engineering teams while providing technical leadership through design reviews, code reviews, and mentoring.

Connaissances

Azure Databricks

dbt

Python

PySpark

SQL

Delta Lake

Unity Catalog

Azure

CI/CD

Data Engineering

Earnix

Outils

Kafka

Event Hubs

FastAPI

REST APIs

Power BI

Terraform

Dataiku

MLflow

Description du poste

Coforge is hiring a Senior Data Engineer (Earnix) to design and implement production-grade data pipelines and governed data products using Azure Databricks, dbt, Python/PySpark, SQL, and Delta Lake. You will build Silver/Gold data models for pricing data and ensure scalable Lakehouse governance with Unity Catalog.

The role requires 7+ years of data engineering experience, strong Azure Databricks expertise, and ownership of end-to-end pipelines, with remote collaboration across pricing,