IT Development & Operations- Senior Data Engineer & Senior Data Modeler

eTeam; Kuala Selangor; On-site; MYR 85,000 - 120,000; Full time

Job summary

A leading data solutions firm is seeking a Senior Data Engineer and Data Modeler in Kuala Selangor, Malaysia. The role involves leading a small team to architect data solutions, develop ETL processes, and troubleshoot data quality issues. Ideal candidates should have over 8 years of experience, strong skills in SQL, Python, and tools like Talend, with expertise in data architecture and streaming technologies. A customer-oriented mindset and strong communication skills are essential for this position.

Qualifications

• 8+ years of experience in building and architecting data solutions.
• 3+ years of experience developing ETL processes using SQL or tools like Talend.
• Experience in delivering data using streaming technologies (Kafka, Kinesis).

Responsibilities

• Lead a team to understand requirements and design technical solutions.
• Analyze and optimize large databases.
• Troubleshoot data quality issues and implement best practices.

Skills

Leadership ability

Problem solving

Customer service orientation

Team player

Organization and detail-oriented

Excellent communication

Tools

Talend

AWS

SQL

Python

Kafka

Cloudera

Job description

IT Development & Operations - Senior Data Engineer & Senior Data Modeler

You are

(personality / competencies etc)

• Able to lead a small team (Team Lead)
• Passionate and possess strong ownership in area of work.
• Understand customers, business needs and expectation, constantly seek improvement in area of work and drive the success of team's work targets and expectation
• Adaptable to flexible working hours.
• Excellent problem solving skills and like new challenges
• Contribute and enhance the improvement require for existing process
• Customer service oriented mindset and able to assess impact of operational problem and advice relevant mitigation
• Good understanding of the end to end value chain, both technical and business
• Team player, responsible, reliable, and take ownership until task are fully completed E2E
• Comply to all the company SOPP & audit

Your responsibilities

• Work closely with Business Analysts, SME's and end users to understand the requirements in order to determine feasibility. Designs technical solutions that are in line with business needs while staying consistent with application architecture.
• Navigate, understand, analyze and leverage existing complex data structures.
• Clarifies and resolves requirements questions.
• Coordinate with vendor to implement best practices and methodologies in mastering data management and in designing physical data models.
• Follows technical guidelines and best practices to ensure a consistently high standard of quality. Makes recommendations for next steps and long-term improvements.
• Investigate and troubleshoot data quality workflows or data quality issues or recurring technical problems.
• Analyze large databases and recommend appropriate optimization for the same
• Resolves defects or deficiencies with completed development, updates documentation, and provides training to support team members on completed development as necessary.

Your Merits

• 8+ years of relevant experience in building and architecting data solutions.
• 3+ years of experience developing ETL process using SQL or tool like Talend, etc
• 2+ years of experience with Business Intelligence tools
• Expert in SQL and high-level languages such as Python or similar scripting language
• Experience with distributed data architectures and their eco-systems (Spark, EMR, Hadoop, Hive)
• Strong hands-on dimensional data modeling and data warehousing skills such as Microfocus Vertica preferable
• Experience in delivering data using Streaming Technologies (Kafka, Kinesis, Spark Streaming, etc.)
• Experience with continuous integration technologies (Jenkins, GitLab)
• Worked with Cloud-based architecture such as AWS
• Practical experience establishing and utilizing data analytics standard methodologies.
• Excellent organizational skills and detail-oriented.
• Exceptional teammate who establishes positive working relationships.
• Good communication skills with the ability to express sophisticated concepts effectively
• Preferable telco industry experience

Tools We Use (Preferable Skill)

• Talend
• Cloudera Data Hub – HBase, Impala, Kudu, Search, Spark, Spark Streaming, Hive on Spark, Cloudera Manager, Cloudera Director, and Cloudera Navigator
• Data Anonymization - Microfcus Voltage with Talend through API calls.
• Data Management – Cloudera Hadoop, Microfocus Vertica & AWS S3
• Data Access & Reporting - Qlik Sense & Denodo
• Infrastructure – On Premise Redhat Virtualization and Cloud AWS
• OS – Linux and Window