Data Engineer

Quantori; Portugal; Teletrabalho; EUR 45 000 - 60 000; Tempo integral

Resumo da oferta

A technology solutions company in Portugal is seeking a Data Engineer to develop intelligent solutions that enhance discovery and address R&D challenges. The role involves designing data pipelines on the Databricks Platform and transforming data using Python and PySpark. Candidates should be familiar with Databricks, proficient in Python and Spark, and possess knowledge of AWS cloud platforms. Join us to make a tangible impact in clinical insights and research!

Qualificações

• Familiarity with the Databricks platform is expected.
• Ability to write robust code with Python is mandatory.
• Proficiency with Spark (PySpark) is necessary.

Responsabilidades

• Design and create pipelines for data ingestion based on the Databricks Platform.
• Prepare data transformations using Python and PySpark library.

Conhecimentos

Databricks platform familiarity

Robust code writing with Python

Proficiency with Spark (PySpark)

Knowledge of cloud platforms: AWS

Descrição da oferta de emprego

Time Zone: The candidate should be available to work till 5 pm PST

Duration:from 3-6 months with possible extension

Job Description:

We are currently looking for a Data Engineer to assist in developing intelligent solutions enabling our customers to accelerate discovery, acquire clinical insights, and address R&D challenges.

Responsibilities:

• Design and create pipelines for data ingestions based on the Databricks Platform.
• Preparing data transformations with Python and PySpark library.

What we expect:

• Familiarity with the Databricks platform
• Ability to write robust code with Python
• Proficiency with Spark (PySpark)
• Knowledge of cloud platforms: AWS

Nice to have:

Knowledge of Lakehouse concept including experience using Delta Lake optimized storage layer.