Key Responsibilities
- Design, develop, and optimize ETL/ELT pipelines using AWS services (Glue, Athena, S3, Redshift, Iceberg).
- Implement monitoring and alerting mechanisms for data workflows to ensure reliability and timely issue resolution.
- Perform database and schema optimization to improve performance, scalability, and cost efficiency.
- Manage data validation, taxonomy updates, and metadata consistency across pipelines and environments.
- Refactor, clean up, and maintain data jobs, tables, and legacy pipelines to align with best practices.
- Oversee infrastructure upgrades such as EKS cluster updates and database cleanup activities.
- Automate job scheduling and parameterization for flexible and efficient data processing.
- Collaborate with cross-functional teams to integrate external datasets and ensure compliance with data governance standards.
- Any other ad-hoc duties as assigned by supervisor.
Requirements
- Strong proficiency in AWS cloud services: Glue, S3, Athena, Redshift, IAM, and EKS.
- Experience in ETL/ELT pipeline design, optimization, and troubleshooting.
- Solid understanding of SQL, Python, and data transformation frameworks.
- Knowledge of data modeling, schema design, and performance tuning.
- Familiarity with DevOps practices, including automation, CI/CD, and monitoring solutions.
- Ability to manage data quality, validation, and governance processes.
- Nice to Have Experience with data cataloging and metadata management tools (e.g., AWS Glue Data Catalog, DataHub).
- Exposure to real-time or near real-time data processing.
- Knowledge of multi-layered data architectures (bronze/silver/gold or raw/staging/production).
Interested applicants, please email your resume to Andre Chua Jing Ming
Email: andrechua@recruitexpress.com.sg
CEI Reg No: R1989053
EA Licence No: 99C4599
Recruit Express Pte Ltd