Big Data Developer

Tech Aalto Pte. Ltd; Singapore; On-site; SGD 90,000 - 140,000; Full time

Job summary

Tech Aalto Pte. Ltd is seeking a Big Data Developer in Singapore with 3–10 years of experience to architect and implement scalable data processing solutions within the Hadoop ecosystem. The role emphasizes Hive, HDFS, Spark, Scala, and PySpark, along with proficiency in data warehousing concepts and big data tooling.

You will work with Shell scripting and Python, and gain exposure to Java, RESTful services, and Maven, while leveraging CI/CD pipelines and cloud deployment practices.

Qualifications

• Strong understanding of concurrent software systems with emphasis on scalability and robustness.
• Expertise designing solutions within the Hadoop ecosystem (Hive, HDFS, Spark).
• Deep knowledge of Spark SQL and PySpark workflows.
• Experience with file formats like Parquet, ORC, Sequence and their use cases.
• Solid understanding of data warehousing concepts and architectures.
• Experience with Unix/Shell scripting and Python.
• Familiarity with HortonWorks distribution and SQL engines (Tez, MR).
• Exposure to Java, RESTful services, and Maven is a plus.

Skills

Hive

HDFS

Spark

Scala

PySpark

Tools

Jenkins

Jira

ADO DevOps

Bitbucket

Azure DevOps

Job description

Big Data Developer

Experience: 3 – 10 Years

Job Requirements

Core Skills & Expertise

• Strong understanding of concurrent software systems, ensuring scalability, maintainability, and robustness.
• Expertise in designing application solutions within the Hadoop ecosystem.
• Deep knowledge of Hive, HDFS, YARN, Spark, Spark SQL, Scala, and PySpark.
• Proficiency in handling HDFS file formats such as Parquet, ORC, Sequence, and their use cases.
• Solid understanding of data warehousing systems.
• Experience in ing languages such as Shell and Python.
• Familiarity with HortonWorks distribution and SQL engines (Tez, MR).
• Exposure to Java, RESTful services, and Maven is a plus.

Automation & Deployment

• Experience in Control-M development and resource monitoring using Grafana.
• Ability to create automation s in Jenkins for builds, test frameworks, and application configurations.
• Hands‑on experience in scalable application deployment using tools like Bitbucket, Jenkins, and Azure DevOps (ADO).

Skill Set

Mandatory Skills:

• Big Data Technologies: Hive, HDFS, Spark, Scala, PySpark

Good to Have:

• Schedulers: Control-M
• ETL Tools: Dataiku
• ing & OS: Unix/Shell ing
• Integration Services: FileIT, MQ and others
• CI/CD Tools: Jenkins, Jira, ADO DevOps suite