Azure Data Engineer

Capgemini Engineering; Bogotá ciudad; Presencial; COP 120.000.000 - 180.000.000; Jornada completa

Descripción de la vacante

Capgemini Engineering in Bogotá invites a data engineer to design and optimize scalable data pipelines using Python, PySpark, and Azure data services.

You will collaborate with Data Science teams, implement CI/CD, ensure data quality and security, and contribute to AI initiatives in a dynamic, global engineering environment.

Formación

• Bachelor’s degree or higher in a related field such as systems engineering, computer science, software engineering, or data science.
• Proficient in Python, PySpark, FastAPI, and PyTest.
• Experience with Microsoft Azure data services (Databricks, Data Factory, Functions, Storage, Key Vault, Logic Apps, DevOps).
• Strong SQL and REST API skills; version control with Git.

Responsabilidades

• Design, develop, and maintain scalable, high-performance data pipelines.
• Build ETL/ELT processes using Python, PySpark, SQL, Azure Databricks, and Azure Data Factory.
• Implement integration mechanisms for structured and unstructured data from multiple sources.
• Develop storage and processing solutions based on Data Lake and Lakehouse architecture.
• Ensure data quality, consistency, security, and availability.
• Optimize distributed processing workflows for large volumes of data.
• Implement data governance, monitoring, and observability standards.
• Design and implement solutions using Azure Databricks, Azure Data Factory, Azure Functions, Azure Storage Accounts, Azure Key Vault, and Azure Logic Apps.
• Automate deployments via CI/CD pipelines using Azure DevOps.
• Monitor and resolve incidents related to data processes in production environments.
• Apply security and access management best practices in cloud environments.
• Implement secure authentication, authorization, and secret management mechanisms.
• Collaborate with Data Science teams to prepare, transform, and make data available for analytical models.
• Participate in Machine Learning, Artificial Intelligence, and NLP initiatives.
• Implement and support data pipelines for forecasting and predictive analytics models.

Conocimientos

Python

PySpark

FastAPI

PyTest

REST APIs

SQL

Git

Educación

Bachelor's degree in Systems Engineering, Computer Science, Software Engineering, Data Science, or related field

Herramientas

Azure Databricks

Azure Data Factory

Azure Functions

Azure Storage Accounts

Azure Key Vault

Azure Logic Apps

Azure DevOps

Descripción del empleo

At Capgemini Engineering, the world leader in engineering services, we bring together a global team of engineers, scientists, and architects to help the world’s mostinnovative companies unleash their potential. From autonomous cars to life-saving robots, our digital and software technology experts think outside the box as they provide unique R&D and engineering services across all industries. Join us for a career full of opportunities. Where you can make a difference. Where no two days arethe same.

Job Description

Your Role

• Design, develop, and maintain scalable, high-performance data pipelines.
• Build ETL/ELT processes using Python, PySpark, SQL, Azure Databricks, and Azure Data Factory.
• Implement integration mechanisms for structured and unstructured data from multiple sources.
• Develop storage and processing solutions based on Data Lake and Lakehouse architecture.
• Ensure data quality, consistency, security, and availability.
• Optimize distributed processing workflows for large volumes of data.
• Implement data governance, monitoring, and observability standards.
• Design and implement solutions using Azure Databricks, Azure Data Factory, Azure Functions, Azure Storage Accounts, Azure Key Vault, and Azure Logic Apps.
• Automate deployments via CI/CD pipelines using Azure DevOps.
• Monitor and resolve incidents related to data processes in production environments.
• Apply security and access management best practices in cloud environments.
• Implement secure authentication, authorization, and secret management mechanisms.
• Collaborate with Data Science teams to prepare, transform, and make data available for analytical models.
• Participate in Machine Learning, Artificial Intelligence, and NLP initiatives.
• Implement and support data pipelines for forecasting and predictive analytics models.

Your Profile

• Bachelor’s degree in systems engineering, Computer Science, Software Engineering, Data Science, or a related field.
• Python, PySpark, FastAPI, PyTest.
• Microsoft Azure: Azure Databricks (Databricks Notebooks, Databricks Workflows), Azure Data Factory, Azure Functions, Azure Key Vault, Azure Storage Accounts, Azure Logic Apps, Azure DevOps.
• REST APIs.
• SQL.
• Git.

Desirable

• Web Scraping.
• Microsoft Excel for exploratory analysis and data validation.
• Data and Analytics Knowledge.
• Design and implementation of modern data architectures.
• Massive data processing and distributed computing.
• Data Warehousing and Data Lakes.
• Machine Learning Engineering fundamentals.
• Business Intelligence and enterprise analytics.
• Forecasting and time-series analysis.
• Credit and financial risk modeling.
• Financial analysis and performance metrics.
• Cash flow analysis and modeling.
• Experience working with Agile methodologies and Jira.

Capgemini is a global business and technology transformation partner, helping organizations to accelerate their dual transition to a digital and sustainable world, while creating tangible impact for enterprises and society. It is a responsible and diverse group of 340,000 team members in more than 50 countries. With its strong over 55-year heritage, Capgemini is trusted by its clients to unlock the value of technology to address the entire breadth of their business needs. It delivers end-to-end services and solutions leveraging strengths from strategy and design to engineering, all fueled by its market leading capabilities in AI, generative AI, cloud and data, combined with its deep industry expertise and partner ecosystem.