Data Engineer

ONL Biz Solutions Sdn Bhd; Kuala Lumpur; On-site; MYR 120,000 - 160,000; Full time

Job summary

A technology solutions company in Kuala Lumpur seeks a Senior Data Engineer to design and implement scalable data architectures for its growing applications. The ideal candidate will have over 5 years of experience in data engineering, a strong foundation in SQL, and expertise in cloud platforms like AWS or GCP. Key responsibilities include leading data architecture, integrating diverse data sources, and collaborating across teams to enhance data access. If you possess a passion for big data and cloud solutions, apply now!

Qualifications

• 5+ years of experience in data engineering or backend roles.
• Strong experience in designing data pipelines and architectures.
• Proficient in SQL and data modeling.
• Hands-on with cloud platforms such as AWS or GCP.

Responsibilities

• Lead the design and implementation of data architecture.
• Analyze ecosystems for data integration points.
• Propose and implement data ingestion and processing architectures.
• Collaborate with teams to ensure data usability.

Skills

Data pipeline development

SQL proficiency

Data modeling

Data governance

Cloud platforms knowledge

Stakeholder management

Tools

Apache Airflow

Spark

AWS

GCP

Azure

Job description

Senior Data Engineer

We are seeking a Senior Data Engineer to lead the design and implementation of a scalable data architecture tailored to our growing application ecosystem. This is a high-impact role that requires a strong foundation in data lake and data warehouse design, along with the ability to evaluate multiple technologies and recommend the best-fit solution for our business needs.

Key Responsibilities

• Analyze the current ecosystem involving multiple source databases (SQL, NoSQL, APIs, etc.) and identify integration points for a unified data platform.
• Evaluate and compare potential data lake and data warehouse solutions (e.g., Hive, Delta Lake, BigQuery, Snowflake, Redshift) based on cost, performance, scalability, and team needs.
• Propose and implement the most appropriate architecture for both batch and real-time data ingestion and processing.
• Design and build efficient, secure, and reliable ETL/ELT pipelines for structured and semi-structured data.
• Ensure the resulting architecture supports data discoverability, query performance, lineage tracking, and governance.
• Collaborate with data analysts, product teams, and backend developers to ensure seamless data access and usability.
• Define and enforce standards around data quality, security, and compliance.
• Stay up to date with industry best practices and emerging technologies in data engineering and cloud infrastructure.

Requirements

• 5+ years of professional experience in data engineering or backend data-intensive roles.
• Proven experience in designing and implementing data pipelines, data lakes, and data warehouses in cloud or hybrid environments.
• Strong command of SQL, data modeling, and performance tuning.
• Hands-on experience with data pipeline tools such as Apache Airflow, Spark, or equivalents.
• Familiarity with various data storage and processing systems including traditional RDBMS, NoSQL, and streaming platforms (Kafka, Kinesis, etc.).
• Experience in evaluating and selecting data storage and compute platforms (e.g., Hive, Redshift, Snowflake, Delta Lake).
• Experience with cloud platforms such as AWS, GCP, or Azure.
• Excellent communication and stakeholder management skills.

Nice to Have

• Experience with Hive, HDFS, or Hadoop ecosystem components.
• Background in data governance, security frameworks, and compliance (GDPR, HIPAA, etc.).
• Proficiency in scripting or programming languages like Python, Scala, or Java.
• Experience with real-time data processing architectures.