Data Engineer

Hong Kong Unison Limited; Singapore; On-site; SGD 70,000 - 110,000; Full time

Job summary

Unison Consulting is seeking a data engineer in Singapore to design and deploy data warehouses, data lakes and real-time pipelines. You will integrate data silos, build scalable ETL/ELT processes and develop backend APIs to support analytics-driven applications.

The role requires hands-on experience with SQL, Python, SSIS, and cloud platforms (AWS/GCP/Azure). Collaboration with cross-functional teams and CI/CD practices are key elements of the job.

Qualifications

• Proficient in data cleaning and transformation (SQL, pandas, R) to ensure data accuracy.
• Strong ETL experience with SSIS, AWS DMS, Python, AWS Lambda, ECS, AWS Glue, Spring.
• Proficient in designing and managing databases (SQL, PostgreSQL, AWS S3, Athena, MongoDB, MySQL, etc).
• Experience with cloud technologies such as AWS, Azure, Google Cloud.
• Experience building production-grade data pipelines and data integration.
• Knowledge of data modelling, data access, and storage infra like Data Mart/Data Lake/Warehouse.
• Familiar with REST APIs and web data protocols.
• Familiar with big data tools (Hadoop, Spark, Kafka, RabbitMQ).
• Familiar with web scraping tools (BeautifulSoup, CasperJS, Selenium, Node.js).
• Knowledge of data governance, access control and security best practices.
• Comfortable in at least one scripting language (SQL, Python).
• Experience in Windows and Linux development environments.

Responsibilities

• Design, develop and deploy data tables, views and marts in data warehouses, data lake and data virtualization.
• Perform data extraction, cleaning, transformation, and flow; web scraping may be part of scope.
• Design, build, launch and maintain large-scale batch and real-time data pipelines.
• Integrate data silos in scalable and compliant manner.
• Collaborate with Project Manager, Data Architect, Business Analysts, Frontend Developers and Data Analysts.
• Develop backend APIs and work on databases to support applications.
• Work in an Agile CI/CD environment and participate in pair programming and code reviews.

Skills

SQL

Data cleaning

ETL pipelines

Python

Data modelling

REST APIs

Big data

Tools

SSIS

AWS DMS

Python

AWS Lambda

ECS

AWS Glue

Spring

Job description

• Design, develop and deploy data tables, views and marts in data warehouses, operational data store, data lake and data virtualization.
• Perform data extraction, cleaning, transformation, and flow. Web scraping may be also a part of the work scope in data extraction.
• Design, build, launch and maintain efficient and reliable large-scale batch and real-time data pipelines with data processing frameworks.
• Integrate and collate data silos in a manner which is both scalable and compliant.
• Collaborate with Project Manager, Data Architect, Business Analysts, Frontend Developers, Designers and Data Analyst to build scalable data driven products.
• Be responsible for developing backend APIs & working on databases to support the applications.
• Work in an Agile Environment that practices Continuous Integration and Delivery.
• Work closely with fellow developers through pair programming and code review process.

Requirements

• Proficient in general data cleaning and transformation (e.g. SQL, pandas, R, etc) to ensure data accuracy and consistency.
• Proficient in building ETL pipeline (eg. SQL Server Integration Services (SSIS), AWS Database Migration Services (DMS), Python, AWS Lambda, ECS Container task, Eventbridge, AWS Glue, Spring).
• Proficient in database design and various databases (e.g. SQL, PostgreSQL, AWS S3, Athena, mongodb, postgres/gis, mysql, sqlite, voltdb, cassandra, etc).
• Experience in cloud technologies such as GPC, GCC (i.e. AWS, Azure, Google Cloud).
• Experience and passion for data engineering in a big data environment using Cloud platforms such as GPC, GCC (i.e. AWS, Azure, Google Cloud).
• Experience with building production-grade data pipelines, ETL/ELT data integration.
• Knowledge about system design, data structure and algorithms.
• Familiar with data modelling, data access, and data storage infrastructure like Data Mart, Data Lake, Data Virtualisation and Data Warehouse for efficient storage and retrieval.
• Familiar with rest api and web requests/protocols in general.
• Familiar with big data frameworks and tools (eg. Hadoop, Spark, Kafka,RabbitMQ).
• Familiar with W3C Document Object Model and customized web scraping (e.g. BeautifulSoup, CasperJS, PhantomJS, Selenium, Nodejs, etc).
• Familiar with data governance policies, access control and security best practices.
• Comfortable in at least one scripting language (eg. SQL,Python).
• Comfortable in both windows and linux development environments.
• Interest in being the bridge between engineering and analytics.

Unlock job insights

Hirer responsiveness Salary match Number of applicants

Unison Consulting established in the year 2012 in Singapore. We provide technology consulting, systems implementation, and integrations service to Asia Pacific customers. We help transform enterprise data in actionable insights. Our consultants have wide range of functional expertise within Banking and Financial services spectrum, working closely with clients to anticipate, illuminate and overcome complex business challenges.

Our objective is to be a one-stop solution provider all enterprise business intelligence and data warehousing needs of organizations.

Our mission is to consistently deliver our services to our clients in a cost-effective way without compromising the quality.

Our current clients include - DBS, OCBC, Standard Chartered Bank and Maybank.

Unison Consulting established in the year 2012 in Singapore. We provide technology consulting, systems implementation, and integrations service to Asia Pacific customers. We help transform enterprise data in actionable insights. Our consultants have wide range of functional expertise within Banking and Financial services spectrum, working closely with clients to anticipate, illuminate and overcome complex business challenges.

Our objective is to be a one-stop solution provider all enterprise business intelligence and data warehousing needs of organizations.

Our mission is to consistently deliver our services to our clients in a cost-effective way without compromising the quality.

Our current clients include - DBS, OCBC, Standard Chartered Bank and Maybank.