Data Engineer

Avinity Analytics Pte. Ltd.; Petaling Jaya; On-site; MYR 80,000 - 140,000; Full time

Benefits offered by this job

Bonuses

ESOP

Performance benefits

Job summary

Avinity Analytics Pte. Ltd. is hiring a Data Engineer to join a data consultancy that builds end-to-end data solutions for mid-size to enterprise clients. You will progress through learning tracks in data development while delivering data projects that span analytics, automation, and ML initiatives.

The role emphasizes learning, collaboration, and applying latest data technologies to solve real business challenges in a flexible, growth-oriented environment.

Qualifications

• Advanced SQL knowledge with relational databases and query optimization.
• Experience with MSSQL, MySQL, PostgreSQL, and MongoDB.
• Experience building data transformation, structures, metadata, and workflows.
• Experience extracting value from large disconnected datasets.
• Knowledge of message queuing, streaming, and scalable big data stores.
• Proven ability to work in cross-functional teams and manage projects.
• Familiarity with Airflow, dbt, and other data orchestration tools.
• Proficiency in Python or R.

Responsibilities

• Create and maintain data pipeline architecture using Airflow and dbt.
• Wrangle large, complex datasets to meet business requirements.
• Automate processes and redesign infrastructure for scale.
• Build data infrastructure for ETL across diverse sources (SQL, cloud).
• Develop analytics models using the data pipeline to inform metrics.
• Collaborate with Executives, Product, Data and Design teams for data needs.

Skills

SQL

Analytics

Cross-functional collaboration

Python

R

Data modeling

Tools

Airflow

dbt

Luigi

Hadoop

Spark

Kafka

Snowflake

BigQuery

Job description

DATA ENGINEER at a Data Consultancy & Data Solutions Provider

JOB OVERVIEW

Join Avinity Analytics where we focus on providing you with the best training and education in data engineering, as well as relevant data projects that will help you gain the necessary experience to advance your career.

There are multiple vacancies available for senior data engineers with 2 to 4 years of experience, to learn and execute data projects at a basic and advanced level. You will be developing essential data engineering know-how in our 3 year Data Development Program, whilst executing data solutions for mid-size to enterprise clients. Your data engineering skills will be applied in data analytics, data automation, and machine learning & Ai projects.

Working at a data consultancy with a flexible working environment and a continuous learning culture, you will gain significant improvements in your data engineering skillsets. Through solving real business challenges with data, you will be implementing the latest data software and technologies to drive data adoption and innovation.

If you have a great interest to be involved in a variety of data projects and would like to take your data skills to the next level, you are encouraged to apply.

RESPONSIBILITIES

• Create and maintain optimal data pipeline architecture via scheduling and authoring tools such as Airflow & dbt.
• Wrangling large, complex data sets that meet functional / non-functional business requirements.
• Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, and re-designing infrastructure for greater scalability.
• Build the data infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL including Google, Microsoft & AWS cloud data/serverless technologies.
• Build analytics models that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency, and other key business performance metrics.
• Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.

REQUIREMENTS

• Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as query optimization.
• Strong analytic skills related to working familiarity with a variety of databases - MSSQL, MySQL, PostgreSQL, MongoDB.
• Build processes supporting data transformation, data structures, metadata, dependency and workload management.
• A successful history of manipulating, processing, and extracting value from large disconnected datasets.
• Knowledge of message queuing, stream processing, and highly scalable 'big data' data stores.
• Strong project management and organizational skills.
• Experience supporting and working with cross-functional teams in a dynamic environment.
• Experience using one or more of the following software/tools:
• Big data storage & transform tools: Hadoop, Spark, Kafka, Snowflake, BigQuery.
• Relational SQL and NoSQL databases.
• Data pipeline and workflow management tools: Luigi, Airflow.
• Object-oriented/object function scripting languages: Python, R.

WHY JOIN US?

• If you're a data enthusiast, there's no better place than here.
• Work in a flexible, dynamic, and merit based work environment.
• Your personalized learning experience through the data development programme.
• Better than average employment benefits to satisfy your human needs.
• Bonuses, ESOP, and other performance benefits for productive employees.