Data Engineer

TXT TECHNICAL PTE. LTD.; Singapore; On-site; USD 60,000 - 100,000; Full time

Job summary

An established industry player is seeking a skilled Data Engineer to design and build scalable data infrastructures. This role involves developing robust ETL pipelines, integrating diverse data sources, and optimizing data models for performance. You'll collaborate with data scientists and analysts to ensure data accessibility for advanced analytics and machine learning applications. If you have a strong background in data engineering and a passion for data-driven solutions, this is an exciting opportunity to make a significant impact in a dynamic environment.

Qualifications

• 2+ years of experience in data engineering or database management.
• Proficiency in relational and NoSQL databases like MySQL and MongoDB.

Responsibilities

• Design and build scalable data infrastructure for analytics.
• Develop and maintain robust ETL pipelines for data processing.

Skills

Data Infrastructure Development

ETL Pipeline Development

Data Integration

Data Modeling

Optimization and Performance Tuning

Collaboration

Data Governance

Automation

Troubleshooting and Maintenance

Education

Bachelor’s degree in Computer Science

Equivalent professional experience

Tools

Apache Airflow

Talend

Snowflake

Redshift

BigQuery

Git

Apache Spark

Hadoop

Job description

Key Responsibilities:

• Data Infrastructure Development: Design and build scalable, efficient, and secure data infrastructure to support advanced analytics and reporting systems.
• ETL Pipeline Development: Develop and maintain robust ETL (Extract, Transform, Load) pipelines to process data from various sources, ensuring high-quality, consistent, and accurate data.
• Data Integration: Integrate multiple internal and external data sources, ensuring seamless data flow across platforms.
• Data Modeling: Create and maintain data models and schemas that ensure the efficient storage and retrieval of data.
• Optimization and Performance Tuning: Enhance the performance and scalability of data pipelines, optimizing for speed and reliability.
• Collaboration: Work closely with data scientists, business analysts, and other teams to ensure that data is organized and accessible for analysis, reporting, and machine learning applications.
• Data Governance: Implement data quality control processes to maintain data integrity and consistency.
• Automation: Automate data workflows and systems to ensure efficiency and reduce manual work.
• Troubleshooting and Maintenance: Monitor the health of data systems, diagnose issues, and maintain optimal data system performance.

Qualifications:

• Education: Bachelor’s degree in Computer Science, Information Systems, Engineering, or a related field (or equivalent professional experience).
• Experience:2+ years of experience in data engineering, database management, or a similar field.
  Experience working with cloud platforms (AWS, Google Cloud, or Azure).
  Proficiency in relational and NoSQL databases (MySQL, PostgreSQL, MongoDB, Cassandra).
  Solid experience with ETL processes and tools (e.g., Apache Airflow, Talend, or custom-built pipelines).
  Familiarity with data warehousing solutions such as Snowflake, Redshift, or BigQuery.
  
• Technical Skills:Strong proficiency in programming languages such as Python, Java, or Scala.
  Expertise in SQL for data querying, optimization, and performance tuning.
  Experience with data processing frameworks like Apache Spark or Hadoop.
  Version control experience with tools like Git.