Data Engineer - Debezium CDC & Python Pipelines

PT ABISHAR TECHNOLOGIES INDONESIA; Jakarta Utara; On-site; IDR 162,000,000 - 198,000,000; Full time

Job summary

PT ABISHAR TECHNOLOGIES INDONESIA is seeking an experienced Data Engineer to monitor Debezium CDC connectors and optimize Python-based data processing pipelines. The role involves supporting Solace message flows, Oracle On-Prem data delivery, and resolving pipeline bottlenecks within a Kubernetes environment.

The ideal candidate has 4+ years in data engineering, with at least 2 years on Debezium and Python, and 1+ year in Kubernetes.

Qualifications

• Minimum 4 years experience in Data Engineer.
• Debezium: 2+ years with CDC connector configuration and source connectors (Oracle, PostgreSQL, SQL Server).
• Python: 2+ years developing data pipelines, consumer applications, async programming, and error handling.
• Kubernetes: 1+ years with pod management, deployments, services, ConfigMaps, secrets, resource limits, and troubleshooting.
• RDBMS: 2+ years with SQL query handling, data loading (SQL*Loader, INSERT/UPDATE scripts), and performance tuning.

Responsibilities

• Monitor and maintain Debezium CDC connectors capturing database changes.
• Troubleshoot and optimize Python consumer applications processing Solace messages.
• Troubleshoot and support transformed data into Oracle On-Premises ODS.
• Troubleshoot and resolve pipeline failures and performance bottlenecks.
• Support Kubernetes deployment, scaling, and health checks for consumer services.
• Maintain runbooks and operational documentation.

Skills

Python development

Async programming

Troubleshooting

Data processing

Tools

Debezium

Kubernetes

Oracle

PostgreSQL

SQL Server

SQL*Loader

Job description

PT ABISHAR TECHNOLOGIES INDONESIA is seeking an experienced Data Engineer to monitor Debezium CDC connectors and optimize Python-based data processing pipelines. The role involves supporting Solace message flows, Oracle On-Prem data delivery, and resolving pipeline bottlenecks within a Kubernetes environment.

The ideal candidate has 4+ years in data engineering, with at least 2 years on Debezium and Python, and 1+ year in Kubernetes.