Lead Data Engineer: Databricks Analytics Platform

EPAM Systems; Colombia; On-site; COP 120,000,000 - 180,000,000; Full time

Benefits offered by this job

Healthcare benefits

Paid time off

Upskilling & certification courses

LinkedIn Learning access

Global career opportunities

Volunteer opportunities

Job summary

EPAM Systems in Colombia is seeking a Lead Data Engineer to build and evolve a next-generation analytics platform with Databricks and scalable data products. You will combine hands-on delivery with technical leadership across ingestion, modeling, governance, and AI-enabled engineering practices.

The role requires 5+ years in data engineering, 1+ year of technical leadership, strong Spark/PySpark, Databricks production experience, advanced SQL, data governance, and solid CI/CD skills, plus

Qualifications

• 5+ years of data engineering experience
• 1+ years of technical leadership experience guiding engineering standards and best practices
• Strong platform delivery experience building enterprise analytics solutions end to end
• Advanced Apache Spark and PySpark expertise for large-scale processing
• Strong Databricks production experience with performance optimization and operational excellence
• Advanced SQL skills for transformation, modeling, and analytics consumption
• Strong data governance knowledge including security, quality, and lineage practices
• Solid CI/CD experience and Data Engineering SDLC best practices
• Strong communication skills to lead technical discussions with cross-functional stakeholders
• Upper-Intermediate English proficiency (B2) for day-to-day collaboration

Responsibilities

• Design scalable data ingestion, transformation, and data product frameworks
• Define Databricks best practices, including Medallion Architecture, optimization, governance, and operational standards
• Build batch, near real-time, and streaming pipelines using Databricks capabilities
• Deliver end-to-end data products that support analytics use cases across domains
• Develop trusted analytical datasets, semantic models, and governed consumption layers for reporting
• Implement data quality, lineage, monitoring, and observability practices across pipelines
• Enable self-service analytics through reusable, business-ready datasets and platform features
• Lead proof-of-concepts and evaluate emerging capabilities across the Databricks ecosystem
• Define AI SDLC practices and AI-assisted development patterns to accelerate delivery
• Partner with engineering, analytics, product, and business stakeholders and mentor engineers on best practices

Skills

Databricks

Apache Spark

PySpark

SQL

Data governance

CI/CD

Leadership

English communication

Tools

Databricks DL

Job description

EPAM Systems in Colombia is seeking a Lead Data Engineer to build and evolve a next-generation analytics platform with Databricks and scalable data products. You will combine hands-on delivery with technical leadership across ingestion, modeling, governance, and AI-enabled engineering practices.

The role requires 5+ years in data engineering, 1+ year of technical leadership, strong Spark/PySpark, Databricks production experience, advanced SQL, data governance, and solid CI/CD skills, plus