Principal Big Data Engineer (Contract Role)

QUESS SINGAPORE; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

QUESS SINGAPORE seeks an experienced data engineering leader to architect and operate scalable data warehouses and both batch and real-time pipelines (Flink streaming, Hive/HDFS batch).

You will own a multi-cluster data platform, governance, monitoring and reliability improvements, while mentoring engineers and collaborating with product and business teams to deliver dependable data solutions.

Qualifications

• Bachelor’s degree or equivalent practical experience.
• 8+ years in software or data engineering, 4+ years on big data platforms.
• Hands-on production experience with Flink for real-time processing.
• Experience designing multi-tenant data platforms with access control and audit.
• Proficient Java (Spring) with knowledge of Python/Scala/Go a bonus.
• Strong SQL skills; experience with MySQL/Tidb or similar.
• Experience building real-time data services where latency matters.
• Knowledge of data modelling, ETL/ELT, batch and streaming architectures.
• Ability to evaluate trade-offs and mentor engineers.

Responsibilities

• Lead architecture and development of scalable data warehouses and data pipelines (Flink streaming, Hive/HDFS batch).
• Own design and operation of a big data development platform with multi-cluster management (YARN).
• Build real-time user-facing data services (transactions, billing, analytics).
• Design data ingestion systems with redundancy, heartbeat and quotas.
• Define data models, tagging systems and feature datasets for analytics.
• Establish data quality, observability and governance standards and dashboards.
• Plan fault drills and performance tests to reduce risk and boost throughput.
• Improve processing performance and platform stability; scale wisely.
• Provide technical guidance and code reviews; mentor engineers.
• Collaborate with business/product/engineering to translate requirements into data solutions.

Skills

Flink

Hive/HDFS

YARN

Java/Spring

SQL

Python/Scala/Go

Data modelling

ETL/ELT

Monitoring

Distributed systems

Education

Bachelor’s degree in Computer Science or related field

Tools

Spark

Kafka

Trino

Redis

Job description

Key Responsibilities

• Lead the architecture and development of scalable data warehouses and batch and real-time data pipelines (e.g. Flink-based streaming and Hive/HDFS-based batch processing).
• Own the end-to-end design and operation of a big data development platform, including multi-cluster resource management (e.g. YARN), access and permission controls, audit trails and unified authentication.
• Build and maintain real-time user-facing data services, such as transaction, billing and profit-and-loss analytics, with strict accuracy, latency and availability targets.
• Design reliable data ingestion and collection systems for external data sources, including primary/backup links, heartbeat monitoring, quota monitoring and fallback strategies.
• Define data models, tagging systems and feature datasets that support user segmentation, analytics and personalisation.
• Establish standards for data quality, observability, monitoring and governance, including unified monitoring dashboards and alerting.
• Plan and run fault drills and joint performance and stress tests to find bottlenecks, reduce operational risk and prepare the platform for peak-traffic events.
• Improve processing performance, resource efficiency and platform stability, and make sound scaling decisions.
• Provide technical guidance, conduct design and code reviews, and mentor engineers.
• Collaborate with business, product and engineering teams to translate requirements into dependable data solutions.

Job Requirements

Essential

• Bachelor’s degree in Computer Science, Information and Computing Science, Mathematics, Statistics or a related field, or equivalent practical experience.
• At least 8 years of professional software or data engineering experience, including at least 4 years building and operating big data platforms in production.
• Hands-on experience in production with Flink for real-time processing, and with Hive, HDFS and YARN (or equivalent distributed storage, query and resource-management technologies).
• Proven experience designing and running a multi-tenant data or big data development platform, covering resource and cluster management, permissions, auditing and authentication.
• Strong proficiency in Java (including Spring / Spring Cloud microservices); working knowledge of Python, Scala or Go is an advantage.
• Strong SQL skills, including query optimisation, with experience in relational and distributed databases (e.g. MySQL, TiDB).
• Demonstrated experience building real-time data services for transactional or financial data, where accuracy and timeliness are critical.
• Track record of improving system reliability through monitoring, fault drills, failover design and performance or stress testing.
• Solid understanding of data modelling, ETL/ELT, batch and streaming architectures, and data quality practices.
• Ability to evaluate technical trade-offs, set engineering standards and mentor other engineers.

Advantageous

• Experience in payments or other regulated financial environment.
• Experience with Spark, Kafka, Trino or similar technologies.
• Experience with Redis, message queues, gRPC and distributed-systems patterns (locking, idempotency, consistency).
• Experience introducing micro-frontend or modular architectures for internal platforms.
• Experience building tagging or feature datasets for segmentation and recommendation use cases.

Disclaimer: The company is committed to ensuring theprivacy and security of your information. By submitting this form, youconsent to the collection, processing, and retention of the information youprovide. The data collected (which may include your contact details,educational background, work experience and skills) will be used solely for thepurpose of evaluating your qualifications for the position you're applying for.Your data will be stored securely and retained for the duration necessary tofulfill our hiring process. If you are not selected for the position, your datawill be kept on file for a limited period in case future opportunities arise.You have the right to access, correct, or delete your data at any time bycontacting us at Quess Singapore | A Leading Staffing Services Provider inSingapore (quesscorp.sg)

"This is in partnership with the Employment and Employability Institute Pte Ltd ("e2i").

e2i is the empowering network for workers and employers seeking employment andemployability solutions. e2is serves as a bridge between workers andemployers, connecting with workers to offer job security through job-matching,career guidance and skills upgrading services, and partnering employers toaddress their manpower needs through recruitment, training, and job redesignsolutions. e2iis a tripartite initiative of the National Trades Union Congress set up to support nation-wide manpower and skills upgradinginitiatives.By applying for this role, you consent to QuesscorpSingapore’s PDPA and e2i’s PDPA. By submitting your application for this position, youconsent to the collection, use, and disclosure of your personal data by Quesscorp and its recruitment partners, including Employment and Employability Institute Pte Ltd, for the purpose of i) employment facilitationand ii) data and statistical analysis.You may find out more informationon the PDPA policies at Quesscorp website and[https://www.e2i.com.sg/pdpa/].