Senior Data Engineer — Real-Time Pipelines & Lakehouse

Sequoia Connect; Ciudad de México; Presencial; MXN 600.000 - 900.000; Jornada completa

Descripción de la vacante

Sequoia Connect is seeking a Senior Data Engineer to join our global IT partner team. You will design and optimize scalable data pipelines, build lakehouse architectures with Delta tables, and develop RESTful APIs for enterprise data exchange.

You will work with Azure Data Factory, Databricks, Synapse, and real-time streaming technologies to ensure data quality, security, and compliant data platforms while collaborating with data scientists and business stakeholders on high-impact projects.

Formación

• Proficiency in SQL and database systems (Oracle, SQL Server, MySQL).
• Strong programming skills in Python, PySpark, PL/SQL, Java or Scala.
• Hands-on with Azure cloud services (ADF, Databricks, ADLS, Synapse, Event Hubs, Functions, API Management, DevOps).
• Experience with Databricks Lakehouse, Delta Lake and Delta Tables.
• API development, RESTful services, microservices architecture.
• Experience with real-time data ingestion and streaming (Kafka, Event Hubs, Kinesis).
• Familiarity with Spark, Hadoop and distributed processing frameworks.
• Hands-on with OpenShift, Kubernetes, Docker; containerized deployments.
• Experience with Airflow and Azure Data Factory for workflows.
• Understanding of data governance, security and compliance.

Responsabilidades

• Design, develop, and maintain scalable ETL/ELT pipelines for batch and real-time data processing.
• Build data models, Delta Tables, and Lakehouse architectures for analytics.
• Develop RESTful APIs and data services for data exchange across systems.
• Implement real-time data ingestion frameworks with streaming tech.
• Utilize Azure services (ADF, Databricks, ADLS, Synapse) for cloud-native data solutions.
• Develop Databricks Spark apps for large-scale transformations.
• Ensure data quality, governance, security, and compliance across platforms.
• Collaborate with data scientists, analysts, and stakeholders to deliver data solutions.
• Troubleshoot and optimize pipeline performance and reliability.
• Support DataOps and CI/CD practices for data deployment.

Conocimientos

SQL proficiency

Python/PySpark

Java/Scala

Azure cloud

Delta Lake

API development

Data processing

Data governance

OpenShift/Kubernetes

Airflow

Herramientas

Azure Databricks

Azure Data Factory

Synapse Analytics

Kubernetes

Docker

OpenShift

Apache Airflow

Descripción del empleo

Sequoia Connect is seeking a Senior Data Engineer to join our global IT partner team. You will design and optimize scalable data pipelines, build lakehouse architectures with Delta tables, and develop RESTful APIs for enterprise data exchange.

You will work with Azure Data Factory, Databricks, Synapse, and real-time streaming technologies to ensure data quality, security, and compliant data platforms while collaborating with data scientists and business stakeholders on high-impact projects.