Senior Data Engineer

PT BCI CENTRAL INDONESIA; Daerah Khusus Ibukota Jakarta; On-site; USD 30.000 - 50.000; Full time

Job summary

A leading technology company is seeking a Sr. Data Engineer for their global Data Enrichment squad in Jakarta. The role involves developing ETL workflows using Apache Airflow and AWS, improving data flow operations, and mentoring junior team members. Ideal candidates should have extensive experience with AWS, Apache Airflow, and database technologies like PostgreSQL.

Qualifications

• 5+ years of experience as a full-time Data Engineer.
• Hands-on experience with Apache Airflow in past 2 years.
• Experience with CI/CD pipelines for ETL processes.

Responsibilities

• Develop and maintain ETL workflows using Apache Airflow and AWS.
• Operate and monitor data flows in production environments.
• Guide junior team members in testing practices.

Skills

SQL

AWS

Python

Tools

Apache Airflow

PostgreSQL

OpenSearch

Job description

About the Role: As a Sr. Data Engineer for our global Data Enrichment squad, you will be responsible for implementing data flows using Apache Airflow and AWS, incorporating functionalities such as external APIs, AI/ML components, and GenAI. You will work closely with your ETL Lead and Product Owner to translate customer and stakeholder requirements into ETL data flow features, then build and operate them. You will also improve the operations of ETL data flows by enhancing monitoring capabilities, automating error recovery, and scaling processes.

Responsibilities:

1. Develop and maintain ETL workflows using Apache Airflow and AWS services.
2. Take ownership of complex ETL tasks and AWS Lambda functions.
3. Design ETL pipelines and review work from junior team members.
4. Contribute to building secure and scalable data solutions.
5. Operate and monitor data flows in production environments, supporting junior colleagues as needed.
6. Improve the resilience, reliability, and performance of ETL systems.
7. Automate recurring operations and optimize performance.
8. Guide junior team members in testing practices and setting up automated tests.
9. Promote and model best practices for documentation across the team.

Requirements:

1. 5+ years of experience as a full-time Data Engineer.
2. Hands-on experience delivering projects using Apache Airflow in the past 2 years.
3. Proficiency with AWS.
4. Experience with PostgreSQL and/or OpenSearch.
5. Strong SQL skills demonstrated through past roles and projects.
6. Experience implementing and improving CI/CD pipelines for ETL processes.
7. Solid understanding and several years of experience working with Python in a production environment.

Note: The question 'How many years' experience do you have as a Data Engineer?' appears to be part of the application process, not the job description itself.