Data Platform Engineer: Spark, Python & Delta Lake

Jobtailor; Lisboa; Presencial; EUR 45 000 - 65 000; Tempo integral

Resumo da oferta

Jobtailor in Lisbon seeks an experienced Data Engineer to design, develop and optimize data pipelines using Spark and Spark Streaming.

You will write scalable Python following PEP 8, work with SQL for data modeling and analytics, and use Databricks for notebooks, clusters and Delta Lake.

Collaborating with diverse teams, you will apply DevOps practices with GitHub Actions and CI/CD, with a strong focus on delivery and continuous improvement.

Qualificações

• Strong Python (OOP) skills and adherence to PEP 8.
• Extensive experience with Spark and Spark Streaming.
• Proficient in SQL for data modeling and analytics.
• Hands-on Databricks experience: notebooks, clusters, Delta Lake.
• Familiarity with DevOps; GitHub Actions and CI/CD tooling.

Responsabilidades

• Develop, optimize and monitor data pipelines using Spark and Spark Streaming.
• Write scalable Python code following PEP 8 with unit and integration tests.
• Model data with SQL for analytics and integrate with analytical systems.
• Use Databricks for notebooks, clusters and Delta Lake.
• Collaborate with cross-functional teams and communicate effectively.

Conhecimentos

Python (OOP)

Spark

SQL

Databricks

Delta Lake

GitHub Actions

CI/CD

Azure / GCP

Unit & Integration Testing

Agile / Delivery

Descrição da oferta de emprego

Jobtailor in Lisbon seeks an experienced Data Engineer to design, develop and optimize data pipelines using Spark and Spark Streaming.

You will write scalable Python following PEP 8, work with SQL for data modeling and analytics, and use Databricks for notebooks, clusters and Delta Lake.

Collaborating with diverse teams, you will apply DevOps practices with GitHub Actions and CI/CD, with a strong focus on delivery and continuous improvement.