Currículum y carta de presentación personalizados, listos para enviar.
Supera los filtros ATS
Descripción de la vacante
A leading consulting firm in Singapore is seeking a Data Engineer to design and develop data pipelines using Azure and Pyspark, and maintain datalakes. The ideal candidate will possess a Bachelor's degree in Computer Science or Engineering and 2 years of relevant experience. Responsibilities include managing data pipelines and integration to ensure efficient data handling. Candidates should have solid knowledge of Azure Synapse and strong problem-solving skills. This position offers an opportunity for professional growth in a dynamic environment.
Formación
2 years of experience in Azure Data engineering, Python, Pyspark or Big Data development.
Knowledge of Azure Synapse analytics for pipelines and orchestration.
Experience in Visualization design with Power BI, including row-level security.
Responsabilidades
Design and develop Pyspark scripts and manage data pipelines.
Implement and maintain a datalake for data storage.
Conduct data integration using database joins as required.
Conocimientos
Azure Data engineering
Python
Pyspark
SQL
Datawarehouse
Power BI
Educación
Bachelor’s Degree in Computer Science or Engineering
Herramientas
Azure DevOps
Hadoop
Descripción del empleo
Data engineer (Azure)
Synapse and Pyspark, Python, Datawarehouse and Azure Data Explorer, Azure Devops
Job Scope
Design, review and development of Pyspark scripts. Testing, troubleshooting of data pipelines, orchestration
Implement and maintain datalake
Establishing connections to source data systems such as on-prem databases, IOT devices, APIs
Managing the collected data in appropriate storage/data-base solutions e.g. file systems, SQL servers, Big Data platforms such as Hadoop, HANA, etc. as required by the specific project requirements.
Design, development of relevant data pipelines using pyspark, copy data activities for batch ingestion
Performing data integration e.g. using database table joins, or other mechanisms at an appropriate level as required by the analysis requirements of the project
Deployment of pipeline artifacts from one environment to the other using Azure Devops
Preferred to have Devops expertise
Skills & Experience
Bachelor’s Degree in Computer Science or Engineering with 2 years of experience in Azure Data engineering, Python, Pyspark or Big Data development
Sound Knowledge of Azure Synapse analytics for pipelines, orchestration, set up
1-2 experience in Visualization design and development with Power BI. Knowledge on row-level security, access control
Sound experience in SQL, Datawarehouse, data marts, data ingestion with Pyspark and Python
Expertise in developing and maintaining ETL processing pipelines in cloud-based platforms such as AWS, Azure, etc. (Azure Synapse or data factory preferred)
Team player with good interpersonal, communication, and problem-solving skills.
Good to have
Data catalog with Purview enabling effective metadata management, lineage tracking, and data discovery
Candidates should demonstrate the ability to leverage Purview to ensure data governance, compliance, and efficient data exploration within Azure environments
Others
Able to work independently on assignment according to agreed schedule without much supervision
Own assignment and take initiative to resolve issues hind completion of assignment. Proactively reach out for help/guidance whenever required.
Consigue la evaluación confidencial y gratuita de tu currículum.
Me encontraba atascado, enviaba solicitudes sin respuesta hasta que empecé a usar JobLeads. Hicieron que mi currículum fuera imposible de ignorar por las empresas.
Sophie Reynolds
La evaluación de currículums de JobLeads me ayudó a solucionar algunos errores fundamentales. ¡Empecé a recibir invitaciones a entrevistas casi inmediatamente!
Daniel Fischer
Con la revisión de currículums de JobLeads, ¡mi currículum pasó rápidamente de ignorado a listo para entrevistas!
Puestos de trabajo similares que vale la pena comparar