Senior Data Engineer: Databricks-powered ETL & Pipelines

Averis Sdn Bhd; Kuala Lumpur; On-site; MYR 120,000 - 240,000; Full time

Benefits offered by this job

Medical benefits

Training and development

Overseas postings

Learning and development programs

Rewards and recognition

Recreational & engagement

Job rotations

Membership subscriptions

Job summary

Averis Sdn Bhd is seeking an experienced Data Engineer in Kuala Lumpur to design, develop, and maintain robust data pipelines. You will work with Python/Java/Scala, Databricks, Spark, and ETL tools to ingest, clean, and store data across multiple sources.

The role emphasizes data quality, security, and collaboration with data scientists for analytics. The ideal candidate has 5+ years in data engineering, strong SQL/NoSQL skills, and hands-on experience with data warehouses like Redshift,

Qualifications

• Bachelor’s degree in Computer Science, Information Technology, or a related field (or equivalent work experience).
• 5+ years of experience in data engineering or a related role.
• Proficiency in programming languages such as Python, Java, or Scala.
• Strong experience with data warehouse technologies (e.g., AWS Redshift, Google BigQuery, Snowflake).
• Hands-on experience with ETL tools and frameworks (e.g., Apache Spark, Apache NiFi).
• Experience with Databricks and its integration with data engineering tasks.
• Knowledge of database systems (SQL and NoSQL) and data modeling.
• Familiarity with data integration and orchestration tools (e.g., Apache Airflow).
• Understanding of data security and compliance standards.
• Excellent problem-solving and communication skills.
• Ability to work independently and collaboratively in a fast-paced environment.
• Strong attention to detail and a commitment to delivering high-quality work.
• Experience with version control systems (e.g., Git).
• Established in 2006, Averis is recognised as a multinational Global Business Services and Solutions provider that specialises in Finance & Accounting, Human Resources, IT Project & Operation, Shipping Documentation, Change Management, Recruitment Process Outsourcing, Corporate Function Services, and Digital.
• We’re currently headquartered in Kuala Lumpur, Malaysia, and have recently expanded to Penang, Dubai, Brazil, and Indonesia. To date, we support our customers across various industries that range from manufacturing to RSPO certified palm, and clean and renewable energy.

Responsibilities

• Collaborate with cross-functional teams to understand data requirements and design efficient data pipelines.
• Develop, implement, and maintain ETL (Extract, Transform, and Load) processes to collect, clean, and store data from various sources.
• Optimize and enhance existing data pipelines for better performance, scalability, and reliability.
• Ensure data quality and integrity through data validation, monitoring, and error handling.
• Troubleshoot and resolve data-related issues in a timely manner.
• Collaborate with data scientists and analysts to provide them with the necessary data sets and tools to perform advanced analytics and reporting.
• Implement data security and privacy best practices to protect sensitive information.
• Stay updated with the latest trends and technologies in data engineering and contribute to the continuous improvement of our data infrastructure.
• Must have hands-on experience with Databricks for building and optimizing data pipelines and performing data engineering tasks.

Skills

Python

Java

Scala

Data engineering

SQL/NoSQL

Airflow

Databricks

Spark

NiFi

Education

Bachelor’s degree in CS/IT or equivalent

Tools

AWS Redshift

BigQuery

Snowflake

Apache Spark

Apache NiFi

Airflow

Git

Job description

Averis Sdn Bhd is seeking an experienced Data Engineer in Kuala Lumpur to design, develop, and maintain robust data pipelines. You will work with Python/Java/Scala, Databricks, Spark, and ETL tools to ingest, clean, and store data across multiple sources.

The role emphasizes data quality, security, and collaboration with data scientists for analytics. The ideal candidate has 5+ years in data engineering, strong SQL/NoSQL skills, and hands-on experience with data warehouses like Redshift,