Senior Data Engineer

InfoConnect Sdn Bhd; Petaling Jaya; On-site; MYR 180,000 - 300,000; Full time

Job summary

InfoConnect Sdn Bhd in Malaysia is seeking a Senior Data Engineer to manage, modernize, and scale our core data infrastructure. You will bridge legacy enterprise data architectures with modern cloud environments, overseeing pipelines across IBM DataStage, SAS ETL, SAP Data Integration, and AWS Glue.

You will build data pipelines and feature stores to support ML and predictive analytics, optimize SQL performance, and collaborate with BI and data governance teams to ensure well-documented,

Qualifications

• 5+ years in Data Engineering, ETL Architecture, and Database Management.
• Hands-on with AWS Glue, PySpark/Spark, and AWS data ecosystems.
• Experience with IBM DataStage, SAS ETL, and SAP Data Integration (BODS/DS).
• Proficient in Python (Pandas, PySpark, Scikit-Learn) for ML data prep and ML pipelines.
• Strong SQL, indexing, schema design (relational, columnar, star schema).
• Nice to have Cognos Analytics or other BI layers; knowledge of data governance.

Responsibilities

• Architect, build, and optimize scalable ETL/ELT pipelines in AWS Glue, Lambda, and Redshift/Athena.
• Maintain and optimize IBM DataStage and SAS ETL jobs; extract from SAP Data Integration environments to cloud platform.
• Design, build, and maintain data pipelines for ML model training, feature stores, and automated scoring.
• Identify bottlenecks, optimize complex SQL queries, and ensure efficient data movement across databases and cloud storage.
• Collaborate with Enterprise BI & Data Governance teams to ensure outputs are well-documented and migration-ready.

Skills

Data engineering

ETL architecture

Database design

Python for ML

SQL optimization

Adaptability

Tools

AWS Glue

PySpark

IBM DataStage

SAS ETL

SAP Data Services

IBM Cognos Analytics

Job description

We are looking for a Senior Data Engineer to manage, modernize, and scale our core data infrastructure. In this role, you will bridge legacy enterprise data architectures and modern cloud environments . Managing pipelines across IBM DataStage, SAS ETL, SAP Data Integration, and AWS Glue. You will also build the data pipelines and feature stores required to support our growing machine learning and predictive analytics initiatives.

Key Responsibilities

• Cloud Pipeline Development: Architect, build, and optimize scalable ETL/ELT pipelines in AWS Glue (PySpark/Python), AWS Lambda, and Redshift/Athena.
• Legacy & ERP Integration: Maintain and optimize existing IBM DataStage and SAS ETL jobs, and orchestrate data extractions from SAP Data Integration environments into our cloud data platform.
• Data Pipelines for ML: Collaborate with data science and AI teams to design, build, and maintain data pipelines for machine learning model training, feature stores, and automated model scoring.
• Performance & Optimization: Identify pipeline bottlenecks, optimize complex SQL queries, and ensure efficient data movement across production databases and cloud storage.
• Cross-Team Collaboration: Partner closely with the Enterprise BI & Data Governance Specialist to ensure pipeline outputs are structured, documented, and migration- and Cognos-ready for downstream reporting and analytics use.

Key Qualifications

• Experience: 5+ years in Data Engineering, ETL Architecture, and Database Management.
• Cloud & Spark: 3+ years of hands-on experience with AWS Glue, PySpark/Spark, and AWS data ecosystems.
• Legacy Enterprise Tools: Hands-on experience with IBM DataStage, SAS ETL (Base SAS / SAS Data Integration Studio), and SAP Data Integration (BODS / SAP Data Services).
• Machine Learning Support: Proficient in Python (Pandas, PySpark, Scikit-Learn) with experience preparing data for ML workflows and MLOps pipelines.
• Database Mastery: Advanced SQL skills, database optimization, indexing strategies, and schema design (relational, columnar, star schema).
• Nice to Have: Exposure to IBM Cognos Analytics or other BI reporting layers, and familiarity with data governance/archiving concepts.
• Adaptability: Willingness and enthusiasm to learn new tools, platforms, and techniques as our data engineering and cloud stack evolves.

Unlock job insights

Hirer responsiveness Salary match Number of applicants

Your application will include the following questions:

• How many years' experience do you have as a Data Engineer?
• Which of the following types of qualifications do you have?
• Do you have experience using ETL Tools?
• How many years experience in Data Engineering do you have?

Computer Software & Networking 51-100 employees

At InfoConnect, we are leaders in Enterprise Information Management (EIM) solutions.With Malaysia as our headquarters and a strong presence in Indonesia, we have a 30-year track record in serving clients across the ASEAN region in various industries, spanning from financial services, public sector, manufacturing, logistics, and more. Our goal is to transform how organizations manage, capture and utilize their data to drive business success.From document and data capture to comprehensive content management, we provide advanced solutions that drive efficiency, streamline processes, and enable data-driven decisions. Our clients span, and we are committed to delivering unparalleled value to all.

At InfoConnect, we are leaders in Enterprise Information Management (EIM) solutions.With Malaysia as our headquarters and a strong presence in Indonesia, we have a 30-year track record in serving clients across the ASEAN region in various industries, spanning from financial services, public sector, manufacturing, logistics, and more. Our goal is to transform how organizations manage, capture and utilize their data to drive business success.From document and data capture to comprehensive content management, we provide advanced solutions that drive efficiency, streamline processes, and enable data-driven decisions. Our clients span, and we are committed to delivering unparalleled value to all.