Software Consultant (Data Engineer)

INTELLECT MINDS PTE. LTD.; Singapore; On-site; SGD 90,000 - 150,000; Full time

Job summary

INTELLECT MINDS PTE. LTD. is seeking a data engineering specialist in Singapore to design and operate scalable data pipelines on AWS using PySpark, Python, and modern orchestration tools.

The role emphasizes building ETL/ELT workflows, serverless solutions, and IaC practices with a focus on data lake architectures. The ideal candidate will have strong experience with AWS Glue, Lambda, Step Functions, Terraform, and CI/CD, and will collaborate with stakeholders to deliver robust data solutions in

Qualifications

• Hands-on PySpark and Python for data engineering.
• Experience developing ETL pipelines using AWS Glue.
• Proficiency with AWS Step Functions for workflow orchestration.
• Experience building serverless solutions using AWS Lambda.
• Hands-on experience with Terraform for Infrastructure as Code (IaC).
• Experience implementing CI/CD pipelines using GitLab, GitHub Actions, Jenkins, or similar.
• Good understanding of AWS services, data lakes, IAM, S3, CloudWatch, and monitoring.

Responsibilities

• Design, develop, and maintain scalable data pipelines using PySpark on AWS.
• Build and orchestrate ETL/ELT workflows using AWS Glue and AWS Step Functions.
• Develop serverless applications and automation using AWS Lambda.
• Write clean, efficient Python/PySpark code following engineering best practices.
• Provision and manage cloud infrastructure using Terraform (IaC).
• Implement and maintain CI/CD pipelines to automate code deployment, testing, and infrastructure changes.
• Monitor, troubleshoot, and optimize data pipelines for performance, reliability, and cost efficiency.
• Collaborate with business stakeholders to deliver data solutions.
• Follow DevOps, security, and coding standards throughout the engagement.

Skills

PySpark

Python

Data modeling

Data engineering

Tools

AWS Glue

AWS Step Functions

AWS Lambda

Terraform

CI/CD pipelines

S3/IAM/CloudWatch

Job description

This role requires strong data Engineering& analysis expertise acrossSQL, Data modelling, DBT, Airflow,and Lakehouse/data lake architecture, with hands-on experience querying large-scale distributed datasets directly from data lakes.

Banking/financial services experience, exposure to ledger/ transactional/ regulatory data, support for migration/modernization program, familiarity, with data governance / metadata. regulatory controls.

Key Responsibilities

• Design, develop, and maintain scalable data pipelines using PySpark on AWS.
• Build and orchestrate ETL/ELT workflows using AWS Glue and AWS Step Functions.
• Develop serverless applications and automation using AWS Lambda.
• Write clean, efficient, and maintainable Python/PySpark code following engineering best practices.
• Provision and manage cloud infrastructure using Terraform (Infrastructure as Code).
• Implement and maintain CI/CD pipelines to automate code deployment, testing, and infrastructure changes.
• Monitor, troubleshoot, and optimize data pipelines for performance, reliability, and cost efficiency.
• Collaborate with business stakeholders to deliver data solutions.
• Follow DevOps, security, and coding standards throughout the engagement.

Required Skills

• Strong hands-on experience with PySpark andPythonfor data engineering.
• Experience developing ETL pipelines using AWS Glue.
• Proficiency with AWS Step Functions for workflow orchestration.
• Experience building serverless solutions using AWS Lambda.
• Hands-on experience with Terraform for Infrastructure as Code (IaC).
• Experience implementing CI/CD pipelines using tools such as GitLab, GitHub Actions, Jenkins, or similar.
• Good understanding of AWS services, data lakes, IAM, S3, CloudWatch, and monitoring.