Big Data Engineer

AXPARA SOLUTIONS PTE. LTD.; Singapore; On-site; SGD 70,000 - 100,000; Full time

Job summary

A technology solutions company in Singapore is seeking a highly skilled Big Data Engineer to design and develop scalable big data solutions. Responsibilities include optimizing data pipelines using SQL and Python, collaborating with cross-functional teams, and managing cloud infrastructure. Candidates should have strong AWS experience and expertise in data governance. This position offers an exciting opportunity to lead key data initiatives.

Qualifications

• Strong proficiency in SQL for data modeling, querying, and transformation.
• Advanced Python development skills with an emphasis on data engineering use cases.
• Hands-on experience with Terraform for cloud infrastructure provisioning.

Responsibilities

• Design, implement, and optimize robust data pipelines and ETL/ELT workflows.
• Collaborate with Data Engineers, Analysts, and engineering teams.
• Build synchronous and asynchronous data APIs for data consumption.

Skills

SQL

Python

Terraform

AWS services

CI/CD tools

Data modeling

Tools

Docker

Kubernetes

Apache Spark

Hive

Kafka

Job description

Job Description:

• We are seeking a highly skilled and motivated Big Data Engineer to join our data team. The ideal candidate will play a key role in designing, developing, and maintaining scalable big data solutions while providing technical leadership. This role will also support strategic Data Governance initiatives, ensuring data integrity, privacy, and accessibility across the organization.

Key Responsibilities:

• Design, implement, and optimize robust data pipelines and ETL/ELT workflows using SQL and Python.
• Collaborate closely with Data Engineers, Analysts, and cross-functional engineering teams to meet evolving data needs.
• Build synchronous and asynchronous data APIs for downstream systems to consume the data.
• Deploy and manage infrastructure using Terraform and other
• Infrastructure-as-Code (IaC) tools.
• Develop and maintain CI/CD pipelines for deploying data applications and services.
• Leverage strong experience in AWS services (e.g., S3, Glue, Lambda, RDS, Lake
• Formation) to support scalable and secure cloud-based data platforms.
• Handle both batch and real-time data processing effectively.
• Apply best practices in data modeling and support data privacy and data protection initiatives.
• Implement and manage data encryption and hashing techniques to secure sensitive information.
• Ensure adherence to software engineering best practices including version control, automated testing, and deployment standards.
• Lead performance tuning and troubleshooting for data applications and platforms.

Required Skills & Experience:

• Strong proficiency in SQL for data modeling, querying, and transformation.
• Advanced Python development skills with an emphasis on data engineering use cases.
• Hands-on experience with Terraform for cloud infrastructure provisioning.
• Proficiency with CI/CD tools, particularly GitHub Actions.
• Deep expertise in AWS cloud architecture and services.
• Demonstrated ability to create and evaluate ERDs and contribute to architectural decisions.
• Strong communication.

Preferred Qualifications:

• Experience with big data technologies such as Apache Spark, Hive, or Kafka.
• Familiarity with containerization tools (e.g., Docker) and orchestration platforms (e.g., Kubernetes) .
• Solid understanding of data governance, data quality, and security frameworks.