Data Engineer

USER EXPERIENCE RESEARCHERS PTE. LTD; Região Norte; Presencial; BRL 150 000 - 210 000; Tempo integral

Resumo da oferta

USER EXPERIENCE RESEARCHERS PTE. LTD is seeking a data engineering professional to analyze data needs, design efficient collection channels, and build scalable reporting solutions.

You will work with Python, PySpark, SQL, and BI tools to deliver dashboards and insights for clients. The role requires hands-on experience with big data technologies, ETL processes, and familiarization with AI-assisted development and automation practices.

Qualificações

• Proven experience delivering projects with Waterfall/Agile methods.
• Strong skills in SQL, data modelling and analysis are required.
• Hands-on with Python, PySpark, Linux, and ETL tools (Informatica).
• Experience with reporting/visualization tools like SAP BO or Tableau is essential.

Responsabilidades

• Analyze client data needs and document requirements.
• Refine data collection by migrating to more efficient channels.
• Plan, design and implement data engineering jobs and reporting solutions.
• Develop test plans and scripts for system testing and UAT support.
• Build reports and dashboards per user requirements.
• Collaborate with client's technical teams for deployment and adoption of solutions.
• Ensure IT solutions run smoothly and meet service levels.
• Support production issues.
• Apply AI coding tools to support development, testing, and code reviews.
• Use automation to improve development, build/test/deploy activities.
• Leverage AI to improve code quality and documentation.

Conhecimentos

SQL

Data modelling

Data analysis

Python

PySpark

Linux

ETL tools

Shell scripting

Hive

Impala

Cloudera Data Platform

Plotly

Informatica

Tableau

Denodo

Ferramentas

Cloudera AI

Plotly

Informatica

SAP BO

Tableau

Denodo

Python

Shell scripting

Hive/Impala

Cloudera Data Platform

Descrição da oferta de emprego

Key Responsibilities:

• Analyze theclient'sdata needs and document the requirements.
• Refine data collection/consumption by migrating data collection to more efficient channels.
• Plan, design and implement data engineering jobs and reporting solutions to meet the analytical needs.
• Develop test plans and scripts for system testing, and support user acceptance testing.
• Build reports and dashboards according to user requirements.
• Work with theclient'stechnical teams to ensure smooth deployment and adoption of new solutions.
• Ensure the smooth operations and service levels of IT solutions.
• Support production issues.
• Apply AI coding tools such as Copilot and Coder AI to support development, testing, and code reviews.
• Use automation to improve development efficiency. This includes build, test, and deployment activities.
• Leverage AI assistance to improve code quality. This covers refactoring, defect resolution, and documentation.
• Demonstrate willingness to adopt new AI and automation tools where relevant to work outcomes.

What We Are Looking For

• Good understanding and completion of projects using Waterfall/Agile methodology.
• Working experience using Cloudera AI (Machine Learning), Plotly, and data engineering AI tool stack.
• Strong SQL, data modelling, and data analysis skills are a must.
• Hands-on experience in big data engineering jobs using Python, PySpark, Linux, and ETL tools like Informatica.
• Hands-on experience in a reporting or visualization tool like SAP BO and Tableau is a must.
• Hands-on experience in DevOps deployment and data virtualisation tools like Denodo will be an advantage.
• Hands-on experience in scripting such as Python and shell scripting will be an advantage.
• Track record in implementing systems using Hive, Impala, and Cloudera Data Platform will be preferred.
• Good understanding of analytics and data warehouse implementations.
• Ability to troubleshoot complex issues ranging from system resources to application stack traces.
• Track record in implementing systems with high availability, high performance, and high security hosted across data centres or hybrid cloud environments will be an added advantage.
• Passion for automation, standardization, and best practices.
• AI and automation experience will be an advantage.