Senior Data Engineer

Capgemini Engineering; Colombia; Presencial; COP 90.000.000 - 130.000.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Stable Employment

Learning & Development

Language Training

Health Coverage

Insurance Protection

Wellness Perks

Descripción de la vacante

Capgemini Engineering Colombia is seeking a data engineer to design, build, and optimize data pipelines for analytics. You will work with Databricks, Spark, and Delta Lake to ensure data quality and scalable architectures, collaborating with data scientists and analysts on ML workloads.

The role emphasizes data governance, security, and production support. Responsibilities include integrating data from diverse sources, supporting CI/CD workflows, and contributing to data migration and

Formación

• Hands-on experience with data engineering or big data development.
• Strong knowledge of SQL (Oracle/SQL Server) and Python.
• Experience with Databricks, Spark, Delta Lake, and data lake architectures.
• Familiarity with cloud platforms (Azure, AWS, or GCP) and data orchestration tools.
• Experience with CI/CD pipelines and version control (Git).

Responsabilidades

• Design, develop, and support data replication and integration solutions using HVR.
• Build scalable data pipelines using Databricks (Spark, Delta Lake).
• Develop and optimize ETL/ELT processes for structured and unstructured data.
• Ensure data quality, integrity, and performance optimization on large datasets.
• Collaborate with data scientists and analysts to enable advanced analytics and ML workloads.
• Integrate data from various sources including databases and APIs.
• Monitor and troubleshoot data pipelines and production issues.
• Support CI/CD pipelines and DevOps practices for data engineering workflows.

Conocimientos

Data engineering

Big data development

SQL

Python

Spark (PySpark/Scala)

CI/CD

Herramientas

Databricks Platform

Apache Spark

Delta Lake

Git

Cloud platforms (Azure/AWS/GCP)

Descripción del empleo

At Capgemini Engineering, the world leader in engineering services, we bring together a global team of engineers, scientists, and architects to help the world’s most innovative companies unleash their potential. From autonomous cars to life-saving robots, our digital and software technology experts think outside the box as they provide unique R&D and engineering services across all industries. Join us for a career full of opportunities. Where you can make a difference. Where no two days are the same

Key Responsibilities

• Design, develop, and support data replication and integration solutions using HVR
• Design, build, and maintain scalable data pipelines using Databricks (Spark, Delta Lake)
• Develop and optimize ETL/ELT processes for structured and unstructured data
• Work with large datasets to ensure data quality, integrity, and performance optimization
• Implement data models and transformations for analytics and reporting
• Collaborate with data scientists and analysts to enable advanced analytics and ML workloads
• Integrate data from multiple sources including databases, APIs, and streaming systems
• Optimize Spark jobs for performance tuning and cost efficiency
• Implement data governance, security, and access controls
• Monitor and troubleshoot data pipelines and production issues
• Support CI/CD pipelines and DevOps best practices for data engineering workflows
• Support data migration and modernization initiatives.
• Ensure data quality, governance, security, and compliance standards.
• Create operational documentation, runbooks, and support procedures.
• Participate in production support, issue resolution, and performance tuning activities.

Technical Skills

• Hands-on experience with data engineering or big data development
• Strong knowledge of:
• SQL / Oracle / SQL Server
• Web-based applications and APIs (REST/SOAP)
• Strong experience with:
• Databricks Platform
• Apache Spark (PySpark/Scala)
• SQL & Python
• Experience with Delta Lake and data lake architecture
• Hands-on experience with cloud platforms (Azure, AWS, or GCP)
• Familiarity with data orchestration tools (Azure Data Factory, Airflow, etc.)
• Knowledge of data warehousing concepts (Star schema, Snowflake schema)
• Experience with version control (Git) and CI/CD pipelines
• Strong understanding of data pipeline optimization and performance tuning

What We Offer

• Stable Employment: Permanent contract offering long-term job security.
• Learning & Development: Access to a wide range of online training platforms and professional development resources.
• Language Training: Weekly virtual English classes and conversation sessions with certified instructors. Online Courses for different languages.
• Health Coverage: Comprehensive prepaid medical and dental plans.
• Insurance Protection: Life and accident insurance for peace of mind.
• Wellness Perks: Discounts and benefits through fitness and technology partnerships.

About Capgemini

At Capgemini Colombia, we aim to attract the best talent and are committed to creating a diverse and inclusive work environment, so there is no discrimination based on race, sex, sexual orientation, gender identity or expression, or any other characteristic of a person. All applications welcome and will be considered based on merit against the job and/or experience for the position.