Senior Data Engineer - Big Data Pipelines & AI

UARROW PTE. LTD.; Singapore; On-site; SGD 150,000 - 210,000; Full time

Job summary

UARROW PTE. LTD. in Singapore seeks a Senior Data Engineer to design, develop and optimize scalable data pipelines using Spark/PySpark, SQL and Python, while collaborating with architects, data scientists and engineers.

You will build robust ETL/ELT workflows, ingest data from diverse sources, tune performance and implement data models, data warehouses and data lakes. Experience with AWS, Hadoop ecosystems and modern AI tech is valued.

Qualifications

• Bachelor's or Master's degree in Computer Science, Information Technology, Engineering or a related field.

Responsibilities

• Design, develop and maintain scalable batch and real-time data pipelines using Apache Spark/PySpark, SQL and Python.
• Develop and optimize ETL/ELT pipelines for large-scale data processing.
• Build data ingestion solutions using databases, APIs, files and streaming platforms.
• Work with AWS services including S3, Glue, EMR, Redshift, Kinesis, Lambda and DynamoDB.
• Develop and optimize data processing solutions using Hadoop, Hive, Spark, Kafka, Cloudera and Databricks.
• Perform SQL and Spark performance tuning and optimize large-scale data processing workloads.
• Design and implement data models, data warehouses and data lake solutions.
• Develop CI/CD pipelines and support containerized deployments using Docker, Kubernetes and OpenShift.
• Integrate Generative AI and NLP capabilities into enterprise data applications where required.
• Develop solutions using LLM frameworks, RAG, vector databases and AI APIs.
• Collaborate with solution architects, data scientists, software engineers and business stakeholders to deliver production-ready solutions.
• Participate in system design, development, testing, deployment and production support.

Job description

UARROW PTE. LTD. in Singapore seeks a Senior Data Engineer to design, develop and optimize scalable data pipelines using Spark/PySpark, SQL and Python, while collaborating with architects, data scientists and engineers.

You will build robust ETL/ELT workflows, ingest data from diverse sources, tune performance and implement data models, data warehouses and data lakes. Experience with AWS, Hadoop ecosystems and modern AI tech is valued.