Data Engineer: Cloud Data Pipelines & ETL

Activate Interactive; Singapore; On-site; SGD 90,000 - 170,000; Full time

Benefits offered by this job

Competitive compensation

Medical coverage & flexi benefits

Career development & training

Job summary

Activate Interactive Pte Ltd is seeking a data engineering specialist for its Co-Development unit based in Singapore. You will design, build and maintain scalable data pipelines and data warehouses to support analytics across government projects.

The role involves cloud-based data processing, ETL/ELT development, web data extraction, and collaboration with cross-functional teams to deliver robust data-driven applications. This is a fixed-term contract for 1 year in Singapore.

Qualifications

• . Proficient in database design and various databases (e.g. SQL, PostgreSQL, AWS S3, Athena, mongodb, postgres/gis, mysql, sqlite, voltdb, cassandra, etc)
• 2. Experience in cloud technologies such as GPC, GCC (i.e. AWS, Azure, Google Cloud)
• 3. Experience and passion for data engineering in a big data environment using Cloud platforms such as GPC, GCC (i.e. AWS, Azure, Google Cloud)
• 4. Experience with building production-grade data pipelines, ETL/ELT data integration
• 5. Knowledge about system design, data structure and algorithms
• 6. Familiar with data modelling, data access, and data storage infrastructure like Data Mart, Data Lake, Data Virtualisation and Data Warehouse for efficient storage and retrieval
• 7. Familiar with rest api and web requests/protocols in general
• 8. Familiar with big data frameworks and tools (eg. Hadoop, Spark, Kafka,RabbitMQ)
• 9. Familiar with W3C Document Object Model and customized web scraping (e.g. BeautifulSoup, CasperJS, PhantomJS, Selenium, Nodejs, etc)
• 10. Familiar with data governance policies, access control and security best practices
• 11. Comfortable in at least one scripting language (eg. SQL,Python)
• 12. Comfortable in both windows and linux development environments.
• 13. Interest in being the bridge between engineering and analytics

Responsibilities

• Design, develop and deploy data tables, views and marts in data warehouses, operational data store, data lake and data virtualization
• Perform data extraction, cleaning, transformation, and flow. Web scraping may be also a part of the work scope in data extraction
• Design, build, launch and maintain efficient and reliable large-scale batch and real-time data pipelines with data processing frameworks
• Integrate and collate data silos in a manner which is both scalable and compliant
• Collaborate with Project Manager, Data Architect, Business Analysts, Frontend Developers, Designers and Data Analyst to build scalable datadriven products
• Be responsible for developing backend APIs & working on databases to support the applications
• Work in an Agile Environment that practices Continuous Integration and Delivery
• Work closely with fellow developers through pair programming and code review process
• Proficient in general data cleaning and transformation (e.g. SQL, pandas, R, etc) to ensure data accuracy and consistency
• Proficient in building ETL pipeline (eg. SQL Server Integration Services (SSIS), AWS Database Migration Services (DMS), Python, AWS Lambda, ECS Container task, Eventbridge, AWS Glue, Spring)

Skills

SQL

Python

ETL/ELT

Data Warehousing

Big Data

REST APIs

AWS

Spark

Tools

Selenium

CasperJS

Job description

Activate Interactive Pte Ltd is seeking a data engineering specialist for its Co-Development unit based in Singapore. You will design, build and maintain scalable data pipelines and data warehouses to support analytics across government projects.

The role involves cloud-based data processing, ETL/ELT development, web data extraction, and collaboration with cross-functional teams to deliver robust data-driven applications. This is a fixed-term contract for 1 year in Singapore.