Data Engineer

Offgamers; Kuala Lumpur; On-site; MYR 60,000 - 110,000; Full time

Job summary

Offgamers in Kuala Lumpur is seeking a Data Engineer to develop, maintain, and optimize ETL/ELT pipelines using AWS Glue, DMS, and Spark. You will build ingestion into the data lake, model data, ensure quality, and monitor production workflows.

Fresh graduates with 0–3 years of experience are welcome; proficiency in SQL and Python is expected, with a willingness to learn AWS cloud data technologies and AI-assisted practices. Collaboration with stakeholders is essential.

Qualifications

• Bachelor’s degree in a relevant field as stated.
• 0–3 years of relevant experience; fresh graduates welcome with demonstrable project ownership.
• Proficiency in SQL and working knowledge of Python.
• Solid understanding of ETL/ELT, data modeling, and data warehouse concepts.
• Strong attention to data quality, accuracy, and detail.
• Strong analytical and problem-solving skills; troubleshoot data workflows.
• Good communication and documentation skills with stakeholders.
• Willingness to learn AWS cloud data technologies and AI-assisted practices.
• Ability to work independently and take ownership with support as needed.
• Nice-to-have experience with AWS services, Spark, BI tools, and data governance.

Responsibilities

• Develop, maintain, and optimize ETL/ELT pipelines using AWS Glue (Spark), workflows, and triggers.
• Build ingestion for structured and semi-structured data from databases, APIs, and files into the S3 data lake.
• Develop data models and curated datamarts; maintain mappings based on business rules.
• Implement data validation and quality checks; monitor production pipelines and alerts.
• Investigate and resolve pipeline failures and data quality incidents via root-cause analysis.
• Optimize SQL queries, Athena scans, and Glue configurations for performance and cost.
• Analytics & Reporting: Build and maintain Tableau dashboards; translate requests into metrics.
• Develop and operate automated reporting; ensure accuracy and monitor discrepancies.
• Platform & Practices: Use AI tooling to accelerate development; document pipelines and mappings.
• Handle data governance, security practices (PII, IAM, KMS), and contribute to engineering standards.

Skills

SQL

Python

ETL/ELT

Data modeling

Data warehouse concepts

Data quality

Analytical thinking

Communication skills

AWS cloud

Ownership

Education

Bachelor's degree in Computer Science, Data Engineering, Information Systems, or a related field

Tools

AWS Glue

AWS DMS

Tableau

PySpark

Apache Iceberg / Delta Lake

KMS / IAM

Git / CI/CD

Job description

• Develop, maintain, and optimize ETL/ELT pipelines using AWS Glue (Spark), Glue workflows, and triggers.
• Build ingestion for structured and semi-structured data from databases (AWS DMS / CDC), APIs, and file sources into the S3 data lake.
• Develop data models and curated datamarts in Athena/Iceberg, maintaining source-to-target mappings based on business rules.
• Implement data validation and quality checks; monitor production pipelines and respond to alerts.
• Investigate and resolve pipeline failures and data quality incidents through structured root-cause analysis.
• Optimize SQL queries, Athena scan volumes, and Glue job configurations for performance and AWS cost efficiency.
• Analytics & Reporting:
  • Build, extend, and maintain Tableau dashboards.
  • Translate stakeholder requests into well-defined metrics, datasets, and reports.
  • Develop and operate automated reporting so business teams receive accurate, timely data.
  • Validate report accuracy and investigate discrepancies raised by business users.
• Platform & Practices:
  • Use AI tooling (e.g., Claude, MCP integrations) to accelerate development, operations, and reporting workflows.
  • Document pipelines, data mappings, processes, and incident resolutions.
  • Handle data responsibly: follow PII, security, and access-control practices (IAM, KMS, scoped datamarts).
  • Contribute to engineering standards, code reviews, and continuous improvement within the Data team.
• Bachelor’s degree in Computer Science, Data Engineering, Information Systems, or a related field.
• 0–3 years of relevant experience. Fresh graduates are welcome — demonstrable project ownership (academic, personal, or internship) is a must-have.
• Proficiency in SQL and working knowledge of Python (depth expected to match experience level).
• Solid understanding of ETL/ELT, data modeling, and data warehouse / data lake concepts.
• Strong attention to data quality, accuracy, and detail.
• Strong analytical and problem-solving skills; able to troubleshoot data workflows to root cause.
• Good communication and documentation skills; able to work with both technical and non-technical stakeholders.
• Willingness to learn AWS cloud data technologies and AI-assisted engineering practices.
• Ability to work independently and take ownership of assigned work, with support scaled to your level.
• Nice to have:
  • Hands-on experience with AWS services such as S3, Glue, Athena, Lambda, or DMS.
  • Experience with Apache Spark / PySpark or open table formats (Apache Iceberg, Delta Lake, Hudi).
  • Experience with BI tools such as Tableau or Power BI.
  • Experience using AI coding tools (Claude, Copilot, Codex) or exposure to MCP / LLM integrations.
  • Exposure to streaming platforms (Kinesis, Kafka) or workflow orchestration tools.
  • Understanding of data governance, PII handling, security, and compliance principles.
  • Familiarity with Git, CI/CD, or Infrastructure as Code.
  • Experience in e-commerce, marketplace, or gaming domains.