Senior ETL Data Engineer: PySpark, Delta Lake & Kinesis

Iris Software, Inc.; Hinoba-an; Presencial; PHP 600.000 - 1.000.000; Jornada completa

Descripción de la vacante

Iris Software, Inc. is seeking a data engineering professional to design and manage scalable data pipelines using PySpark and Databricks across our analytics platforms.

You will contribute to performance-tuned batch and streaming workflows and drive data quality, governance, and architecture improvements. Join a fast-growing IT services company with a focus on Lakehouse architectures, modern data platforms, and AI-assisted development practices.

Formación

• Mandatory skills include PySpark, Databricks Workflows, Delta Lake on Databricks, Amazon Kinesis.

Responsabilidades

• Design scalable data engineering solutions using PySpark and modern distributed data processing frameworks.
• Define data ingestion, transformation, and processing architectures aligned with business and analytical objectives.
• Design and optimize Snowflake or Delta Lake on Databricks solutions to support enterprise-scale data platforms.
• Lead implementation of high-performance batch and streaming data pipelines.
• Define data streaming standards, integration frameworks, and scalable processing patterns.
• Architect workflow orchestration solutions using Apache Airflow or Databricks Workflows.
• Establish monitoring, scheduling, and operational controls for reliable pipeline execution.
• Drive data quality, validation, reconciliation, and governance practices across data engineering solutions.
• Design data engineering solutions following modern Lakehouse architecture principles, data observability practices, and platform engineering standards to improve scalability, reliability, and operational visibility.
• Drive development of business-focused data products by improving data quality, discoverability, usability, documentation, and trusted data consumption across analytical platforms.
• Promote responsible use of AI-assisted engineering capabilities to improve development productivity, testing, documentation, and engineering quality.
• Review data pipeline designs and implementations to ensure adherence to engineering, scalability, and performance standards.
• Troubleshoot complex data processing, workflow, and streaming platform issues through detailed root cause analysis.
• Collaborate with various teams and stakeholders to support end-to-end data platform delivery.

Conocimientos

PySpark

Databricks Workflows

Delta Lake

Amazon Kinesis

Herramientas

Snowflake

Databricks

Apache Airflow

Descripción del empleo

Iris Software, Inc. is seeking a data engineering professional to design and manage scalable data pipelines using PySpark and Databricks across our analytics platforms.

You will contribute to performance-tuned batch and streaming workflows and drive data quality, governance, and architecture improvements. Join a fast-growing IT services company with a focus on Lakehouse architectures, modern data platforms, and AI-assisted development practices.