Data Engineer: Real-Time Pipelines (NiFi, Airflow, Kafka)

Tentacle Tech; Selangor; Presencial; MYR 90.000 - 120.000; Jornada completa

Descripción de la vacante

Tentacle Tech in Selangor, Malaysia is seeking an experienced Data Engineer to lead the design and execution of robust data pipelines and data warehousing for a premier banking project.

You will build and optimize ETL/ELT workflows using NiFi, Airflow, Iceberg, and Kafka, supporting real-time analytics and scalable storage with a focus on data integrity and performance. 3–5 years of relevant experience and Malaysia residency are required.

Formación

• 3–5 years of progressive data engineering experience.
• Experience with ETL/ELT development and data warehousing.
• Operational expertise in NiFi, Airflow, Iceberg, and Kafka.
• Proficient in Python, PowerShell, Bash, and YAML.
• Strong Linux, networking, and distributed storage knowledge.
• Experience with SCD data models and data lake warehousing.

Responsabilidades

• Architect and deploy robust ETL/ELT pipelines and data warehouses for banking platforms.
• Manage distributed streaming and orchestration using NiFi, Airflow, Iceberg, Kafka.
• Develop automation scripts in Python, PowerShell, Bash, and YAML.
• Administer Linux servers, networks, and big data compute environments.
• Design data lake and dimensional warehouse architectures with SCD models.
• Monitor ELK stack logging and real-time analytics, maintain data lineage.
• Troubleshoot pipeline failures and latency issues; produce deployment runbooks.
• Collaborate with cross-functional squads and technical recruiters.

Conocimientos

Analytical problem solving

Stakeholder communication

Linux administration

Data warehousing

ETL/ELT development

Educación

Bachelor’s degree in Computer Science, IT, Software Engineering, Data Analytics or related

Herramientas

Apache NiFi

Apache Airflow

Apache Iceberg

Apache Kafka

ELK Stack

Descripción del empleo

Tentacle Tech in Selangor, Malaysia is seeking an experienced Data Engineer to lead the design and execution of robust data pipelines and data warehousing for a premier banking project.

You will build and optimize ETL/ELT workflows using NiFi, Airflow, Iceberg, and Kafka, supporting real-time analytics and scalable storage with a focus on data integrity and performance. 3–5 years of relevant experience and Malaysia residency are required.