Data Scientist - Big Data & Spark Analytics

Capgemini Singapore Pte Ltd; Singapore; On-site; SGD 120,000 - 180,000; Full time

Job summary

Capgemini Singapore Pte Ltd is seeking a Data Scientist – Big Data to design and implement scalable data solutions using Hadoop ecosystem tools within a collaborative team. You will work with Hive, HDFS, Spark, and Python to deliver robust data pipelines and analytics capabilities.

Join Capgemini to tackle complex data challenges, contribute to DevOps practices, and help clients unlock value from data while adopting modern data architectures.

Qualifications

• 3–10 years of experience in the Big Data ecosystem (Hive, HDFS, YARN, Spark, Spark SQL, PySpark, Scala, Python).
• Proficiency in Shell scripting and Python.
• CI/CD Tools: Jenkins, Azure DevOps (ADO), Bitbucket.
• Monitoring Tools: Grafana or equivalent.
• File formats: Parquet, ORC, Sequence (HDFS formats).
• Strong understanding of concurrent software systems, scalability and robustness.
• Experience in automation and building end-to-end scalable applications.

Responsibilities

• Design and implement robust Big Data solutions using the Hadoop ecosystem.
• Develop data processing pipelines and optimize data workflows.
• Create automation for builds, testing, and configurations; monitor resource usage.
• Contribute to data warehousing and data modeling efforts.
• Implement CI/CD with Bitbucket/Jenkins/ADO for automated deployments.
• Work with integration tools like FileIT or MQ to enable data flows.
• Develop scheduling workflows using Control-M and manage resources.

Skills

Spark (PySpark)

Python

Scala

Shell scripting

CI/CD

Hadoop/HDFS

Data modeling

Tools

Jenkins

Azure DevOps

Bitbucket

Control-M

Dataiku

Job description

Capgemini Singapore Pte Ltd is seeking a Data Scientist – Big Data to design and implement scalable data solutions using Hadoop ecosystem tools within a collaborative team. You will work with Hive, HDFS, Spark, and Python to deliver robust data pipelines and analytics capabilities.

Join Capgemini to tackle complex data challenges, contribute to DevOps practices, and help clients unlock value from data while adopting modern data architectures.