Data Engineer - Databricks

Evalueserve; Colombia; Presencial; COP 100.440.000 - 167.400.000; Jornada completa

Descripción de la vacante

Evalueserve is seeking an experienced Data Engineer to join its Data Analytics practice. You will design and implement scalable ETL/ELT pipelines for data warehouses, lakes, and lakehouses, leveraging Azure Databricks, PySpark, SQL, and Delta Lake.

You will partner with stakeholders to translate business needs into robust data products, and contribute to data architectures, governance, and analytics readiness across enterprise applications.

Formación

• Bachelor’s degree in Engineering, Information Systems, Computer Science, or a related discipline.
• 4+ years of relevant data engineering experience with Azure Databricks, PySpark, SQL, Delta Lake, and Unity Catalog or similar technologies.
• Strong experience in building scalable ETL/ELT pipelines, data models, data warehouses, data lakes, and lakehouse solutions.
• Advanced Python and SQL skills, with knowledge of distributed data processing, performance optimization, and data quality frameworks.
• Experience with GitHub-based source control, CI/CD, automated testing, and Test-Driven Development using Azure DevOps, GitHub Actions, or equivalent tools.
• Understanding of modern data architecture, metadata, documentation, lineage, security, and governance practices.
• Experience using AI-assisted development tools such as Cursor, Claude, GitHub Copilot, or similar solutions.
• Strong problem-solving, communication, and stakeholder management skills.

Responsabilidades

• Design, develop, and automate scalable ETL/ELT pipelines for data warehouse, data lake, and lakehouse environments using Azure Databricks, Apache Spark, Python, and SQL.
• Modernize existing data integration processes and build cloud-native pipelines using Databricks, Delta Lake, Azure Data Factory, or similar technologies.
• Partner with business and technical stakeholders to translate business needs into well-designed data products, platforms, and engineering solutions.
• Develop scalable data architectures and reusable data models to support enterprise applications, business intelligence, advanced analytics, and AI/ML use cases.
• Collaborate with BI and analytics teams to deliver trusted, analytics-ready datasets and enhance reporting environments using Power BI or similar platforms.
• Implement data quality, automated testing, documentation, metadata, lineage, security, and governance practices using capabilities such as Unity Catalog or equivalent frameworks.
• Work with DevOps teams to establish source control, CI/CD, and Test-Driven Development practices using GitHub, Azure DevOps, GitHub Actions, or comparable tools.
• Continuously optimize platform performance, scalability, reliability, operational efficiency, and cloud costs.

Conocimientos

Python

SQL

ETL/ELT design

Data modeling

Data warehousing

Lakehouse concepts

Communication

Educación

Bachelor’s degree in Engineering/Information Systems/Computer Science

Herramientas

Azure Databricks

Delta Lake

Unity Catalog

Azure Data Factory

GitHub Actions

Azure DevOps

PySpark

Power BI

Descripción del empleo

Data Analytics is one of the highest growth practices within Evalueserve, providing you rewarding career opportunities. Established in 2014, the global DA team extends beyond 1000+ (and growing) data science professionals across data engineering, business intelligence, digital marketing, advanced analytics, technology, and product engineering. Our more tenured teammates, some of whom have been with Evalueserve since it started more than 20 years ago, have enjoyed leadership opportunities in different regions of the world across our seven business lines.

Key Responsibilities

• Design, develop, and automate scalable ETL/ELT pipelines for data warehouse, data lake, and lakehouse environments using technologies such as Azure Databricks, Apache Spark, Python, and SQL.
• Modernize existing data integration processes and build cloud-native pipelines using Databricks, Delta Lake, Azure Data Factory, or similar technologies.
• Partner with business and technical stakeholders to translate business needs into well-designed data products, platforms, and engineering solutions.
• Develop scalable data architectures and reusable data models to support enterprise applications, business intelligence, advanced analytics, and AI/ML use cases.
• Collaborate with BI and analytics teams to deliver trusted, analytics-ready datasets and enhance reporting environments using tools such as Power BI or similar platforms.
• Implement data quality, automated testing, documentation, metadata, lineage, security, and governance practices using capabilities such as Unity Catalog or equivalent frameworks.
• Work with DevOps teams to establish source control, CI/CD, and Test-Driven Development practices using GitHub, Azure DevOps, GitHub Actions, or comparable tools.
• Continuously optimize platform performance, scalability, reliability, operational efficiency, and cloud costs.

Required Qualifications

• Bachelor’s degree in Engineering, Information Systems, Computer Science, or a related discipline.
• 4+ years of relevant data engineering experience with Azure Databricks, PySpark, SQL, Delta Lake, and Unity Catalog or similar technologies.
• Strong experience in building scalable ETL/ELT pipelines, data models, data warehouses, data lakes, and lakehouse solutions.
• Advanced Python and SQL skills, with knowledge of distributed data processing, performance optimization, and data quality frameworks.
• Experience with GitHub-based source control, CI/CD, automated testing, and Test-Driven Development using Azure DevOps, GitHub Actions, or equivalent tools.
• Understanding of modern data architecture, metadata, documentation, lineage, security, and governance practices.
• Experience using AI-assisted development tools such as Cursor, Claude, GitHub Copilot, or similar solutions.
• Strong problem-solving, communication, and stakeholder management skills.