Remote Data Engineer: Spark, Databricks & ETL Pipelines

Capgemini Engineering; Aguascalientes; Presencial; MXN 420.000 - 660.000; Jornada completa

Descripción de la vacante

Capgemini Engineering is seeking a MID -Data Engineer- for a REMOTE role. You will design, build, and optimize data pipelines using Databricks and Spark, integrate data from databases, APIs, and streaming sources, and work closely with data scientists to enable advanced analytics.

You will implement data governance, ensure data quality, and support CI/CD for data workflows across cloud platforms such as Azure, AWS, and GCP.

Formación

• Hands-on experience with data engineering or big data development.
• Strong knowledge of SQL / Oracle / SQL Server.
• Strong knowledge of Java / JavaScript.
• Strong knowledge of Web-based applications and APIs (REST/SOAP).
• Strong experience with Databricks Platform.
• Strong experience with Apache Spark (PySpark/Scala).
• Strong experience with SQL & Python.
• Experience with Delta Lake and data lake architecture.
• Hands-on experience with cloud platforms (Azure, AWS, or GCP).
• Familiarity with data orchestration tools (Azure Data Factory, Airflow, etc.).
• Knowledge of data warehousing concepts (Star schema, Snowflake schema).
• Experience with version control (Git) and CI/CD pipelines.
• Strong understanding of data pipeline optimization and performance tuning.

Responsabilidades

• Design, develop, and support data replication and integration solutions using HVR.
• Design, build, and maintain scalable data pipelines using Databricks (Spark, Delta Lake).
• Develop and optimize ETL/ELT processes for structured and unstructured data.
• Work with large datasets to ensure data quality, integrity, and performance optimization.
• Implement data models and transformations for analytics and reporting.
• Collaborate with data scientists and analysts to enable advanced analytics and ML workloads.
• Integrate data from multiple sources including databases, APIs, and streaming systems.
• Optimize Spark jobs for performance tuning and cost efficiency.
• Implement data governance, security, and access controls.
• Monitor and troubleshoot data pipelines and production issues.
• Support CI/CD pipelines and DevOps best practices for data engineering workflows.
• Support data migration and modernization initiatives.
• Ensure data quality, governance, security, and compliance standards.
• Create operational documentation, runbooks, and support procedures.
• Participate in production support, issue resolution, and performance tuning activities.

Conocimientos

Big data

SQL

JavaScript

Python

Data governance

Educación

Bachelor’s degree in Computer Science or related field

Herramientas

Databricks

Apache Spark

Delta Lake

Azure

AWS

GCP

REST APIs

Descripción del empleo

Capgemini Engineering is seeking a MID -Data Engineer- for a REMOTE role. You will design, build, and optimize data pipelines using Databricks and Spark, integrate data from databases, APIs, and streaming sources, and work closely with data scientists to enable advanced analytics.

You will implement data governance, ensure data quality, and support CI/CD for data workflows across cloud platforms such as Azure, AWS, and GCP.