Data Engineer (Apache NiFi, Airflow & Kafka) | Tentacle Tech | Selangor, Malaysia

Tentacle Tech; Selangor; Vor Ort; MYR 90.000 - 120.000; Vollzeit

Zusammenfassung

Tentacle Tech in Selangor, Malaysia is seeking an experienced Data Engineer to lead the design and execution of robust data pipelines and data warehousing for a premier banking project.

You will build and optimize ETL/ELT workflows using NiFi, Airflow, Iceberg, and Kafka, supporting real-time analytics and scalable storage with a focus on data integrity and performance. 3–5 years of relevant experience and Malaysia residency are required.

Qualifikationen

• 3–5 years of progressive data engineering experience.
• Experience with ETL/ELT development and data warehousing.
• Operational expertise in NiFi, Airflow, Iceberg, and Kafka.
• Proficient in Python, PowerShell, Bash, and YAML.
• Strong Linux, networking, and distributed storage knowledge.
• Experience with SCD data models and data lake warehousing.

Aufgaben

• Architect and deploy robust ETL/ELT pipelines and data warehouses for banking platforms.
• Manage distributed streaming and orchestration using NiFi, Airflow, Iceberg, Kafka.
• Develop automation scripts in Python, PowerShell, Bash, and YAML.
• Administer Linux servers, networks, and big data compute environments.
• Design data lake and dimensional warehouse architectures with SCD models.
• Monitor ELK stack logging and real-time analytics, maintain data lineage.
• Troubleshoot pipeline failures and latency issues; produce deployment runbooks.
• Collaborate with cross-functional squads and technical recruiters.

Kenntnisse

Analytical problem solving

Stakeholder communication

Linux administration

Data warehousing

ETL/ELT development

Ausbildung

Bachelor’s degree in Computer Science, IT, Software Engineering, Data Analytics or related

Tools

Apache NiFi

Apache Airflow

Apache Iceberg

Apache Kafka

ELK Stack

Jobbeschreibung

Position Summary:

Driving advanced data engineering, enterprise data warehousing, and real-time data pipeline architecture for a premier banking project in Selangor, Malaysia, Tentacle Tech is actively seeking an accomplished Data Engineer for a renewable contract professional engagement. In this vital technical role, you will take full operational, tactical, and technical ownership of building, optimizing, and scaling robust ETL/ELT data pipelines utilizing Apache NiFi, Airflow, Iceberg, and Kafka. Bringing 3 to 5 years of progressive professional experience in data engineering and financial systems, you will empower top-tier banking institutions to achieve uncompromised data integrity, real-time transaction streaming, and scalable analytics. This prestigious career opportunity offers competitive remuneration, high technical visibility, and professional growth within a premier global IT staffing and digital consulting enterprise.

Detailed Job Description:

As a Data Engineer supporting Tentacle Tech on a renewable contract basis in Selangor, Malaysia, you will take full operational, tactical, and technical ownership of data pipeline development, data lake modelling, log aggregation, and system performance tuning. Your day-to-day responsibilities include authoring complex ETL workflows in Python, PowerShell, and Bash, orchestrating data pipelines with Apache Airflow and NiFi, managing distributed streaming architectures via Apache Kafka and Iceberg, implementing Slowing Changing Dimension (SCD) data models, monitoring ELK stack logging environments, collaborating with multidisciplinary banking squads, and interacting directly with technical recruiters like Saf Suhaimi.

Key Responsibilities:

• Architect, design, develop, test, and deploy robust ETL/ELT data pipelines and enterprise data warehousing architectures for banking platforms.
• Implement and manage distributed data streaming, data integration, and workflow orchestration utilizing Apache NiFi, Airflow, Iceberg, and Kafka.
• Author clean, maintainable, and highly efficient automation scripts and data transformation jobs utilizing Python, PowerShell, Bash, and YAML.
• Configure and administer Linux operating systems, network connectivity, and distributed computing environments for big data workloads.
• Design and maintain enterprise Data Lake architectures, dimensional data warehouses, and Slowing Changing Dimension (SCD) models.
• Administer and monitor log aggregation and real-time search analytics utilizing the ELK Stack (Elasticsearch, Logstash, Kibana).
• Troubleshoot complex data pipeline failures, latency bottlenecks, memory allocation errors, and data synchronization discrepancies.
• Maintain meticulous technical documentation, data lineage maps, architecture blueprints, and deployment runbooks.

Required Qualifications & Skills:

• Bachelor’s degree in Computer Science, Information Technology, Software Engineering, Data Analytics, or related technical discipline.
• Minimum of 3 to 5 years of progressive professional hands‑on experience in data engineering, ETL/ELT development, and data warehousing.
• Strong, demonstrable operational expertise in Apache NiFi, Apache Airflow, Apache Iceberg, and Apache Kafka.
• Comprehensive working knowledge and practical proficiency in programming and scripting languages: Python, PowerShell, Bash, and YAML.
• Deep practical experience in Linux system administration, networking fundamentals, and distributed storage management.
• Solid practical grasp of Data Lake modelling, dimensional warehousing concepts, and SCD data models.
• Exceptional analytical problem‑solving aptitude, system debugging capabilities, and professional stakeholder communication skills.
• Mandatory Malaysian citizenship or current legal residency in Malaysia for professional operations in Selangor.

Nice‑to‑Have Skills:

• Professional certifications such as Certified Data Engineer, Datacenter Professional, or AWS/Azure Data Analytics Specialty.
• Hands‑on familiarity with cloud data platforms, containerization (Docker, Kubernetes), and CI/CD deployment pipelines.
• Prior professional working experience in top‑tier multinational banking institutions, financial technology firms, or major system integrators.
• Experience utilizing enterprise agile tracking platforms (Jira, Confluence) for sprint execution and task coordination.
• Active participation in open‑source data engineering communities, Apache software foundations, or technical forums.

Application Information:

• Salary/Rate: Market Competitive / Renewable Contract Banking Package
• Deadline: Open until filled (Urgent Selangor Data Engineer Opening)
• Notice Period: Immediate to 30 Days Preferred
• Contract Duration: Renewable Contract / Full‑Time Engagement (Selangor, Malaysia)
• Company Website: https://www.tentacletech.com

Recruitment Pro Tip:

When applying for this Data Engineer position through Tentacle Tech, ensure your resume explicitly highlights your Apache NiFi/Airflow/Kafka portfolio, banking data warehouse experience, and confirms your residency in Malaysia.