Data Engineer (Consultant) - DCS

DCI CONSULTANTS PRIVATE LIMITED; Singapore; On-site; SGD 70,000 - 90,000; Full time

Job summary

A leading consulting firm in Singapore is seeking an experienced Data Engineer Consultant to design and implement robust data infrastructures using AWS and Databricks. This senior role involves technical mentorship, collaboration with teams, and hands-on development for efficient data processing. Candidates should have over 4 years of expertise in data engineering and a solid understanding of cloud technologies.

Qualifications

• 4-7 years of data engineering experience with AWS-native implementations.
• Proven experience working on ETL migration projects.
• Hands-on experience implementing Databricks notebooks using PySpark, Python, and SQL.

Responsibilities

• Design and implement enterprise-scale data architectures.
• Develop and maintain ETL/ELT pipelines.
• Provide technical mentorship to junior data engineers.

Skills

AWS cloud platform expertise

Databricks platform proficiency

Data warehouse design

ETL/ELT migration project experience

Real-time streaming technologies

Data exchange platforms and API integration

Clear communication

Active listening

Technical mentorship

Time management

Critical thinking

Education

Bachelor's Degree in Computer Science or related field

Master's Degree in Data Engineering or MBA

Tools

PySpark

Python

SQL

Job description

Job Summary:

We are seeking an experienced Data Engineer Consultant to join our team as a senior individual contributor responsible for designing, implementing, and optimising robust data infrastructure and pipelines. This role offers the opportunity to work hands-on with cutting-edge AWS and Databricks technologies whilst providing technical guidance to team members and collaborating with stakeholders across the organisation on complex data engineering challenges.

Key Responsibilities:

• Data Architecture & Engineering - Design and implement enterprise-scale data architectures, including data lakes, warehouses, and real-time streaming platforms. Develop and maintain ETL/ELT pipelines that efficiently process large volumes of structured and unstructured data from diverse sources. Ensure data quality, governance, and security standards are embedded throughout all data engineering processes.
• Technical Implementation - Hands-on development of Databricks notebooks using PySpark, Python, and SQL for ETL automation. Create and optimise PySpark scripts for efficient data extraction, transformation, and loading from large datasets. Implement custom data manipulation, validation, and error handling solutions to enhance ETL robustness.
• Technical Guidance - Provide technical mentorship to junior data engineers and analysts. Lead code reviews, establish best practices, and drive adoption of modern data engineering tools and methodologies. Collaborate with cross-functional teams including data scientists, analysts, and software engineers to deliver integrated solutions.
• Performance & Optimisation - Monitor and optimise data pipeline performance, implementing solutions for scalability and cost-effectiveness. Conduct testing, debugging, and troubleshooting of data transformation processes. Verify data integrity throughout pipeline stages and resolve complex technical issues.

Essential Skills:

Technical Skills:

• AWS cloud platform expertise (Redshift, S3, Glue, EMR, Kinesis, Lambda)
• Databricks platform proficiency with PySpark, Python, and SQL implementation
• Data warehouse design and implementation (Amazon Redshift, Snowflake on AWS)
• ETL/ELT migration project experience and pipeline development
• Real-time streaming technologies (Kinesis Data Streams, Kafka)
• Data exchange platforms and API integration (AWS Data Exchange, Partner APIs)

Communication Skills:

• Clear written and verbal communication of technical concepts to non-technical stakeholders
• Presentation and reporting abilities for executive audiences
• Active listening to understand business requirements
• Stakeholder management across technical and business teams
• Meeting facilitation for technical design sessions

Leadership Skills:

• Technical mentorship and guidance for junior engineers
• Conflict resolution in cross-functional project environments
• Decision‑making under pressure for critical data system issues
• Knowledge transfer and coaching abilities
• Change management for data platform migrations

Organisational Skills:

• Time management across multiple concurrent data projects
• Resource allocation for optimal pipeline performance
• Multi-tasking coordination of data engineering deliverables
• Attention to detail in data quality and governance processes
• Documentation of technical specifications and system architecture

Problem‑Solving Skills:

• Critical thinking for complex AWS data architecture challenges
• Creative solution development for scalability and performance issues
• Issue escalation and resolution for production data systems
• Adaptability to evolving business and technical requirements
• Strategic planning for AWS data infrastructure roadmaps

Interpersonal Skills:

• Relationship building with business stakeholders and technical teams
• Negotiation of technical requirements and project timelines
• Emotional intelligence in managing diverse project teams
• Cultural awareness in global data initiatives
• Customer service orientation for internal data consumers

Qualifications:

Education Requirements:

• Bachelor's Degree in Computer Science, Engineering, Data Science, or related field
• Master's Degree in Data Engineering, Computer Science, or MBA (preferred)
• Relevant diploma or certificate in AWS cloud technologies or data engineering

Professional Certifications:

• AWS Certified Data Engineer - Associate (preferred)
• AWS Certified Solutions Architect or AWS Certified Big Data - Specialty (preferred)
• Databricks Certified Data Engineer Associate or Professional
• Amazon Redshift or Snowflake certifications
• Agile/Scrum Master certification (preferred)

Experience Requirements:

• 4-7 years of data engineering experience with AWS-native implementations
• Proven experience working as part of a data engineering team on ETL migration projects
• Hands‑on experience implementing Databricks notebooks using PySpark, Python, and SQL for ETL automation
• Demonstrated expertise developing PySpark scripts for efficient data extraction, transformation, and loading from large datasets
• Experience utilising Python for custom data manipulation, validation, and error handling in ETL processes
• Proficiency employing SQL for complex joins, aggregations, and database operations within Databricks environments
• Track record of testing, debugging, and optimising data transformation processes for accuracy and performance
• Experience verifying data integrity throughout pipeline stages and resolving troubleshooting issues
• Proven ability collaborating with cross‑functional teams to align ETL migration tasks with project goals and deadlines
• Experience in government sector data projects and compliance requirements (preferred)
• Experience in HR domain including workforce analytics, payroll systems, and employee data management (preferred)
• Experience providing technical mentorship to junior team members (preferred)

Working Location : Central