Data Engineer - 6 months contract

Zenith Infotech (S) Pte Ltd; Singapore; On-site; SGD 100,000 - 167,000; Full time

Job summary

Zenith Infotech (S) Pte Ltd is seeking an experienced Data Engineer to design and implement data storage, pipelines, and governance using AWS, Databricks, and Informatica IDMC. The role emphasizes data quality, security, and scalable architectures in a hybrid cloud environment.

Ideal candidates have 5–7 years of data engineering experience with a strong background in Python/Java, SQL, and NoSQL. A 6-month contract is offered with opportunities to optimize data workflows and collaborate with

Qualifications

• Bachelor's or master's in computer science, data engineering, or a related field.
• 5–7 years of experience in data engineering with AWS, Databricks, and/or Informatica IDMC.
• Proficiency in Python or Java for building data pipelines.
• Strong SQL and NoSQL knowledge, plus data modeling skills.
• AWS/Databricks certifications are a plus.

Responsibilities

• Design and architect data storage solutions using AWS services and Databricks.
• Integrate Informatica IDMC for metadata management and data governance.
• Create, manage and optimize ETL pipelines for ingesting, processing, and transforming data.
• Monitor and optimize data processing and query performance across AWS and Databricks environments.
• Implement security best practices and automation to ensure data quality and cost efficiency.

Skills

AWS

Databricks

Informatica IDMC

Python

Java

SQL

NoSQL

Data modeling

Education

Bachelor's or master's in CS/Data Eng

Tools

AWS Lambda

Informatica IDMC

Databricks

Job description

Zenith Infotech (S) is a TEch Recruitment firm with 28 years of presence in Singapore, is looking out for experiencedData Engineer with main key skill in Informatica IDMC.

Candidates whom are open for6 months contract roleare welcome to explore this opportunity.

Jobs Scope:

• Design and architect data storage solutions, including databases, data lakes, and warehouses, using AWS services such as Amazon S3, Amazon RDS along with Databricks Delta Lake.
• Integrate Informatica IDMC for metadata management and data cataloging.
• Create, manage, and optimize data pipelines for ingesting, processing, and transforming data using AWS services like AWS Lambda, Databricks for advanced data processing, and Informatica IDMC for data integration and quality.
• Integrate data from various sources, both internal and external, into AWS and Databricks environments, ensuring data consistency and quality, while leveraging Informatica IDMC for data integration, transformation, and governance.
• Develop ETL (Extract, Transform, Load) processes to cleanse, transform, and enrich data, making it suitable for analytical purposes using Databricks Spark capabilities and Informatica IDMC for data transformation and quality.
• Monitor and optimize data processing and query performance in both AWS and Databricks environments, making necessary adjustments to meet performance and scalability requirements. Utilize Informatica IDMC for optimizing data workflows.
• Implement security best practices and data encryption methods to protect sensitive data in both AWS and Databricks, while ensuring compliance with data privacy regulations. Employ Informatica IDMC for data governance and compliance.
• Implement automation for routine tasks, such as data ingestion, transformation, and monitoring, using AWS services like AWS Lambda, Databricks Jobs, and Informatica IDMC for workflow automation.
• Maintain clear and comprehensive documentation of data infrastructure, pipelines, and configurations in both AWS and Databricks environments, with metadata management facilitated by Informatica IDMC.
• Collaborate with cross-functional teams, including data scientists, analysts, and software engineers, to understand data requirements and deliver appropriate solutions across AWS, Databricks, and Informatica IDMC.
• Identify and resolve data-related issues and provide support to ensure data availability and integrity in both AWS, Databricks, and Informatica IDMC environments.
• Optimize AWS, Databricks, and Informatica resource usage to control costs while meeting performance and scalability requirements.
• Stay up-to-date with AWS, Databricks, Informatica IDMC services, and data engineering best practices to recommend and implement new technologies and techniques.

REQUIREMENTS

• Bachelor's or master's degree in computer science, data engineering, or a related field.
• Minimum 5-7 years of experience in data engineering, with expertise in AWS services, Databricks, and/or Informatica IDMC.
• Hands on data migration(e.g. code migration from Information Power Center on premises to cloud version, data migration from Teradata/Oracle/PostgreSQL to Databricks)
• Proficiency in programming languages such as Python or Java for building data pipelines.
• Evaluate potential technical solutions and make recommendations to resolve data issues especially on performance assessment for complex data transformations and long running data processes.
• Strong knowledge of SQL and NoSQL databases.
• Familiarity with data modeling and schema design.
• Excellent problem-solving and analytical skills.
• Strong communication and collaboration skills.
• AWS certifications (e.g., AWS Certified Data Analytics - Specialty), Databricks certifications, and Informatica certifications are a plus