Data Engineer

Agensi Pekerjaan Penta Consultancy Sdn. Bhd; Kuala Lumpur; On-site; MYR 90,000 - 120,000; Full time

Job summary

Agensi Pekerjaan Penta Consultancy Sdn. Bhd is recruiting a Data Engineer to design, develop, and maintain robust ETL/ELT pipelines on Microsoft Azure, leveraging Data Factory, Databricks, and Azure SQL.

You will optimize SQL queries and collaborate with data modelers. The role requires hands-on experience with Azure DevOps for CI/CD, strong SQL tuning skills, and familiarity with Agile tools such as Jira and Confluence.

Qualifications

• 3+ years hands-on ETL/ELT, Databricks, SQL experience.
• Strong SQL tuning and performance optimization.
• Experience with Azure Data Factory and Databricks.
• Knowledge of Azure SQL / Azure SQL Managed Instance.
• Familiarity with Azure DevOps (Repos, CI/CD).
• Understanding of data modeling and normalization principles.
• Experience with Jira and Confluence in Agile environments.
• Vendor management experience and SLAs.

Responsibilities

• Design, develop, and maintain ETL/ELT pipelines on Microsoft Azure (Azure Data Factory, Databricks, Azure SQL / Azure SQL Managed Instance).
• Optimize SQL queries and stored procedures for performance, scalability, and reliability.
• Collaborate with data modelers to understand data structures, normalization, and data quality requirements.
• Implement CI/CD pipelines and deployment strategies using Azure DevOps (Repos, Pipelines, Release Management).
• Monitor, troubleshoot, and improve data pipelines; resolve data quality issues and incidents.
• Support the Data Tech Lead and internal teams with technical discussions and development work.
• Coordinate with Group Office tech team and local IT teams (architects, business analysts, etc.).
• Participate in Agile ceremonies and use tools like JIRA, Confluence, and Kanban boards.

Skills

ETL/ELT

Databricks

SQL

Stored procedures

SQL tuning

Azure Data Factory

Azure DevOps

JIRA

Confluence

Agile

Tools

Azure SQL

Azure SQL Managed Instance

Job description

• Design, develop, and maintain ETL/ELT pipelines on Microsoft Azure (Azure Data Factory, Databricks, Azure SQL / Azure SQL Managed Instance).
• Optimize SQL queries and stored procedures for performance, scalability, and reliability.
• Collaborate with data modelers to understand data structures, normalization, and data quality requirements.
• Implement CI/CD pipelines and deployment strategies using Azure DevOps (Repos, Pipelines, Release Management).
• Monitor, troubleshoot, and improve data pipelines; resolve data quality issues and incidents.
• Support the Data Tech Lead and internal teams with technical discussions and development work.
• Coordinate with Group Office tech team and local IT teams (architects, business analysts, etc.).
• Participate in Agile ceremonies and use tools like JIRA, Confluence, and Kanban boards.

Job Requirements:

• Minimum 3 years of hands-on experience in ETL/ELT, Databricks, SQL, and stored procedures.
• Strong skills in SQL tuning and performance optimization.
• Experience with Azure Data Factory and Databricks.
• Familiarity with Azure SQL or Azure SQL Managed Instance.
• Working knowledge of Azure DevOps (Repos, CI/CD pipelines).
• Understanding of data modeling and normalization principles.
• Experience working in Agile environments using tools like JIRA and Confluence.
• Experience in vendor management, including coordinating with external vendors, managing deliverables, and tracking SLAs.