Remote Senior AI/Big Data Engineer

Oracle; México; Híbrido; MXN 1,294,000 - 1,665,000; Jornada completa

Descripción de la vacante

A leading technology company is seeking a Senior Big Data Engineer in Mexico to design and manage data integration solutions. Candidates should have extensive experience with ETL/ELT processes and expertise in big data frameworks like Apache Spark and Hadoop. Responsibilities include architecting scalable solutions and ensuring data quality and compliance. Competitive salary and benefits available.

Formación

• Proficiency in Python, Java, or Scala for big data processing.
• Strong expertise in Apache Spark, Hadoop, Hive, Flink, or Kafka.
• Hands-on experience with data modeling and data lakes.
• Expertise with ETL/ELT tools like Informatica and Talend.
• Strong hands-on experience with REST, SOAP, and GraphQL APIs.
• Proficiency in batch and real-time integration.

Responsabilidades

• Architect and design scalable big data solutions.
• Develop and maintain ETL/ELT pipelines.
• Integrate data from diverse sources into centralized repositories.
• Implement and manage data lakes and data warehouses.
• Ensure data consistency and compliance with governance standards.

Conocimientos

Python

Java

Scala

Apache Spark

Hadoop

Hive

Flink

Kafka

REST APIs

SOAP APIs

GraphQL APIs

SQL

NoSQL

Herramientas

Informatica

Talend

SSIS

Apache NiFi

dbt

Snowflake

Redshift

BigQuery

Docker

Kubernetes

Descripción del empleo

We are seeking a highly skilled Senior Big Data Engineer to design, develop, and manage enterprise‑grade data integration solutions. The ideal candidate will have extensive experience with ETL/ELT processes, API‑driven integrations, and enterprise data platforms.

Key responsibilities

• Architect, design, and optimize scalable big data solutions for batch and real‑time processing.
• Develop and maintain ETL/ELT pipelines to ingest, transform, and synchronize data from diverse sources.
• Integrate data from cloud applications, on‑prem systems, APIs, and streaming workspaces into centralized data repositories.
• Implement and manage data lakes and data warehouses solutions on cloud infrastructure.
• Ensure data consistency, quality, and compliance with governance and security standards.
• Collaborate with data architects, data engineers, and business stakeholders to align integration solutions with organizational needs.

Core qualifications

• Proficiency in Python, Java, or Scala for big data processing.
• Big Data Frameworks: Strong expertise in Apache Spark, Hadoop, Hive, Flink, or Kafka.
• Hands‑on experience with data modeling, data lakes (Delta Lake, Iceberg, Hudi), and data warehouses (Snowflake, Redshift, BigQuery).
• ETL/ELT Development: Expertise with tools like Informatica, Talend, SSIS, Apache NiFi, dbt, or custom Python‑based frameworks.
• APIs & Integration: Strong hands‑on experience with REST, SOAP, GraphQL APIs, and integration platforms (MuleSoft, Dell Boomi, SnapLogic).
• Data Pipelines: Proficiency in batch and real‑time integration (Kafka, AWS Kinesis/ Azure Event Hub/ GCP Pub/Sub).
• Databases: Deep knowledge of SQL (Oracle, PostgreSQL, SQL Server) and NoSQL (MongoDB, Cassandra, DynamoDB) systems.

Preferred experience

• Expertise with at least one major cloud platform (AWS, Azure, GCP).
• Experience with data services such as AWS EMR/Glue, GCP Dataflow/Dataproc, or Azure Data Factory.
• Familiarity with containerization (Docker) and orchestration (Kubernetes).
• Knowledge of CI/CD pipelines for data engineering.
• Experience with OCI and Oracle Database (including JSON/REST, sharding) and/or Oracle microservices tooling.

How we’ll assess

• Systems design interview: architect a scalable service; justify data models, caching, and failure handling.
• Coding exercise: implement and optimize a core algorithm/data‑structure problem; discuss trade‑offs.
• Code review: evaluate readability, testing, error handling, and security considerations.
• Practical discussion: walk through a past end‑to‑end project, metrics/SLOs, incidents, and learnings.

Qualifications

Career Level - IC4