DATA ENGINEER

TOSS-EX PR PTE. LTD.; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

Koantek is seeking a Senior Data Engineer to design and build robust data pipelines, integrate structured and unstructured data, and develop tools for processing and analysis. You will manage data infrastructure, optimize workflows, and guide data-driven strategies while collaborating with data scientists and stakeholders to deliver scalable, production-ready solutions.

Responsibilities include implementing large-scale data transformations, ensuring CI/CD and DataOps practices, and applying

Qualifications

• Bachelor’s degree in CS, IT, or related field.
• 6+ years as a Data Engineer with multi-cloud experience (AWS/Azure/GCP).
• Proven Databricks experience for large-scale data processing.
• Experience building streaming and batch pipelines; DataOps & CI/CD.
• Strong Spark/Scala, Python, and PySpark proficiency.

Responsibilities

• Design and build robust data pipelines and data infrastructure.
• Integrate structured and unstructured data from multiple sources.
• Develop tools for data processing, analysis, and reporting.
• Guide data-driven strategies and coordinate with data scientists.
• Apply Databricks best practices across cloud environments.

Skills

Spark/Scala

Python

PySpark

Data Modeling

Streaming

Spark SQL

Kafka

StreamSets

ML Frameworks

TensorFlow

PyTorch

CI/CD in Data

Education

Bachelor's degree in Computer Science / IT

Tools

Databricks

AWS

Azure

GCP

MapReduce

Job description

Koantek is a Databricks-exclusive consulting and systems integration partner, backed by Databricks Ventures and a multi-award-winning firm across APJ and global markets. Our deep expertise spans industries such as healthcare, financial services, retail, and SaaS, delivering end-to-end solutions from rapid prototyping to production-scale AI deployments. We deliver tailored solutions that enable businesses to leverage data for growth and innovation. Our team of experts utilizes deep industry knowledge combined with cutting-edge technologies, tools, and methodologies to drive impactful results. By partnering with clients across a diverse range of industries—from emerging startups to established enterprises—we help them uncover new opportunities and achieve a competitive advantage in the digital age.

Description:

As a Senior Data Engineer at Koantek, you will leverage advanced data engineering techniques and analytics to support business decisions for our clients. Your role will involve designing and building robust data pipelines, integrating structured and unstructured data from various sources, and developing tools for data processing and analysis. You will play a pivotal role in managing data infrastructure, optimizing data workflows, and guiding data-driven strategies while working closely with data scientists and other stakeholders.

The impact you will have:

• Guide Big Data Transformations: Implementation of comprehensive big data projects, including the development and deployment of innovative big data and AI applications.
• Ensure Best Practices: Guarantee that Databricks best practices are applied throughout all projects to maintain high-quality service and successful implementation.
• Support Project Management: Assist the Professional Services leader and project managers with estimating efforts and managing risks within customer proposals and statements of work.
• Architect Complex Solutions: Design, develop, deploy, and document complex customer engagements, either independently or as part of a technical team, serving as the technical lead and authority.
• Enable Knowledge Transfer: Facilitate the transfer of knowledge and provide training to team members, customers, and partners, including the creation of reusable project documentation.
• Contribute to Consulting Excellence: Share expertise with the consulting team and offer best practices for client engagement, enhancing the effectiveness and efficiency of other teams.

Minimum qualifications:

Educational Background: Bachelor’s degree in Computer Science, Information Technology, or a related field (or equivalent experience).

Experience:

• 6+ years of experience as a Data Engineer, with proficiency in at least two major cloud platforms (AWS, Azure, GCP).
• Proven experience in designing, developing, and implementing comprehensive data engineering solutions using Databricks, specifically for large-scale data processing and integration projects
• Develop scalable streaming and batch solutions using cloud-native components.
• Perform data transformation tasks, including cleansing, aggregation, enrichment, and normalization, utilizing Databricks and related technologies.
• Experience in applying DataOps principles and implementing CI/CD and DevOps practices within data environments to optimize development and deployment workflows.

Technical Skills:

• Expert-level proficiency in Spark Scala, Python, and PySpark.
• In-depth knowledge of data architecture, including Spark Streaming, Spark Core, Spark SQL, and data modeling.
• Hands-on experience with various data management technologies and tools, such as Kafka, StreamSets, and MapReduce.
• Proficient in using advanced analytics and machine learning frameworks, including Apache Spark MLlib, TensorFlow, and PyTorch, to drive data insights and solutions.

Databricks Specific Skills:

• Extensive experience in data migration from on-premises to cloud environments and in implementing data solutions on Databricks across cloud platforms (AWS, Azure, GCP).
• Skilled in designing and executing end-to-end data engineering solutions using Databricks, focusing on large-scale data processing and integration.
• Proven hands-on experience with Databricks administration and operations, including notebooks, clusters, jobs, and data pipelines.
• Experience integrating Databricks with other data tools and platforms to enhance overall data management and analytics capabilities.

Preferred qualifications:

Certifications:

• Certification in Databricks Engineering (Professional)
• Microsoft Certified: Azure Data Engineer Associate
• GCP Certified: Professional Google Cloud Certified.
• AWS Certified Solutions Architect Professional