Senior Data Engineer

Kruger Inc.; Medellín; Presencial; COP 180.000.000 - 300.000.000; Jornada completa

Descripción de la vacante

Kruger Inc. is seeking a Senior Data Engineer to join the Data Platform team and design, build, and deliver enterprise data products on Microsoft Fabric.

You will provide data solutions for both on-premises and cloud environments, collaborating with business and technical stakeholders to translate requirements into scalable data resources. The role combines strong data engineering with end-user engagement, requiring you to mentor colleagues, participate in architecture discussions, and

Formación

• Bachelor's or Master's degree in Computer Science, Software/Data Engineering, or a related technical field.

Responsabilidades

• Design, develop, and maintain scalable data products using Microsoft Fabric.
• Build and optimize data pipelines, Lakehouses, Warehouses, Semantic Models, and related Fabric components.
• Build and deliver data solution for both on-prem and cloud environments.
• Develop ELT pipelines using SQL, Python, and Spark where appropriate.
• Design source-to-target mappings and transformation logic.
• Optimize data pipelines for scalability, reliability, and performance.
• Ensure solutions follow engineering best practices and enterprise standards.
• Mentor junior engineers or interns and share technical knowledge across the team.

Conocimientos

Python

SQL

Spark PySpark

Lakehouse

Microsoft Fabric

Azure Data Factory

Synapse Analytics

Databricks

Git

CI/CD

DataOps

Data modeling

Stakeholder mgmt

Communication

Problem-solving

Educación

Bachelors/Masters in CS or related field

Herramientas

Git

CI/CD

Azure Data Factory

Synapse Analytics

Databricks

Delta Lake

Descripción del empleo

Kruger is seeking a Senior Data Engineer to join the Data Platform team and play a key role in designing, building, and delivering enterprise data products on Microsoft Fabric, to provide data solutions both on-prem and on cloud.

This role combines strong data engineering expertise with end user engagement. The successful candidate will be able to work directly with stakeholders from both business and technical fields to understand operational processes, discover and profile source data, translate business requirements into scalable technical solutions, and deliver trusted, reusable data products.

The ideal candidate is an experienced engineer who enjoys solving complex data problems and can adapt to fast development requirements, working with modern Lakehouse technologies, and collaborating across business and technical teams.

MAIN RESPONSIBILITIES

• Design, develop, and maintain scalable data products using Microsoft Fabric.
• Build and optimize data pipelines, Lakehouses, Warehouses, Semantic Models, and related Fabric components.
• Build and deliver data solution for both on-prem and cloud environments.
• Develop ELT pipelines using SQL, Python, and Spark where appropriate.
• Design source-to-target mappings and transformation logic.
• Optimize data pipelines for scalability, reliability, and performance.
• Ensure solutions follow engineering best practices and enterprise standards.

Business Discovery & Data Analysis

• Partner with business stakeholders to understand operational processes, KPIs, and reporting requirements.
• Lead discovery sessions to identify business needs and data requirements.
• Profile new data sources to assess data quality, completeness, and business relevance.
• Translate business requirements into scalable technical solutions.
• Identify business rules, data relationships, and transformation requirements.
• Validate datasets with subject matter experts before production deployment.

Data Quality & Governance

• Implement data validation and quality controls throughout data pipelines.
• Investigate and resolve data quality issues.
• Document source systems, mappings, transformations, and business rules.
• Support metadata management, data lineage, and governance initiatives.
• Ensure data products are trusted, reusable, discoverable, and well documented.
• Participate in solution design and technical architecture discussions.
• Contribute reusable engineering patterns, templates, and standards.
• Perform code reviews and promote engineering best practices.
• Implement Git-based source control, CI/CD, and DataOps practices.
• Monitor and continuously improve pipeline reliability and performance.
• Mentor junior engineers or interns and share technical knowledge across the team.

QUALIFICATIONS

• Bachelor's or Master's degree in Computer Science, Software/Data Engineering, or a related technical field.

EXPERIENCE

• 7+ years of experience in Data Engineering, Analytics Engineering, or a related field.
• Proven experience designing and delivering enterprise data solutions.
• Experience working directly with business stakeholders to gather requirements and validate solutions.
• Experience with Git, CI/CD, and DataOps practices.

SKILLS AND ABILITIES

Technical skills

• Strong programming in Python and SQL; hands-on experience with Spark / PySpark for distributed data processing.
• Proven experience building and operating a modern lakehouse or cloud data platform. Experience with Microsoft Fabric, Azure Data Factory, Synapse Analytics, Databricks, Snowflake, or similar modern data platforms.
• Strong understanding of Lakehouse architecture, data modeling, and ELT design.
• Familiarity with Delta Lake or similar modern storage technologies.
• Experience with Git, CI/CD, and DataOps practices.
• Understanding of data quality frameworks and automated testing.
• Demonstrated ability to own projects from discovery through production deployment.
• Excellent communication and stakeholder management skills.
• Ability to facilitate discovery workshops and requirements discussions.
• Strong analytical and problem-solving abilities.
• Ability to explain technical concepts to non-technical audiences.
• Comfortable collaborating with business users, architects, product owners, and engineering teams.

Nice to have

• Manufacturing industry experience.
• Experience with Azure Data Explorer (KQL), Azure Data Factory, Azure ML, and event-driven / streaming ingestion.
• Interest or experience in AI-ready data products, feature stores, vector/RAG layers, MCP servers, and GenAI / LLM agent patterns.
• Knowledge of MES, OEE, SAP ECC/S4, ERP, SCADA, PI Historian, or ISA-95.
• Experience building enterprise data products.
• Familiarity with metadata management, data governance, and data lineage.
• Experience with monitoring, observability, or data quality tools (e.g., Great Expectations).