Senior PySpark Developer — Data Pipelines & Spark Expert (Hybrid Portugal)

Wire IT; Lisboa; Híbrido; EUR 65 000 - 90 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Health Insurance

Vacation 22 days

4 extra days annually

Meal Allowance (Coverflex)

Referral Bonus

Discounts and flex options

Training annual budget

Team-oriented culture

Growth with international company

Challenging projects

Resumo da oferta

Wire IT in Portugal seeks a Senior PySpark Developer to join our hybrid team in Porto or Lisbon. You will design and optimize scalable data pipelines using PySpark and Spark, integrate with Kafka, Parquet/Delta Lake, and cloud platforms.

Strong Python, SQL, and testing skills are essential to ensure reliable data processing. You will work in an agile environment, collaborating with cross-functional teams to deliver high-impact solutions and drive performance improvements across streaming and

Qualificações

• 5+ years of software development or data engineering experience.
• Strong PySpark and Apache Spark experience.
• Advanced Python and SQL skills.
• Experience building scalable batch and streaming data pipelines.
• Deep understanding of Spark architecture, DAGs, and memory tuning.
• Experience with SQL/NoSQL databases, APIs, files, and Kafka streams.
• Knowledge of Parquet, Delta Lake, and Avro.
• Experience with cloud platforms (AWS/Azure/GCP).
• Experience with Databricks, EMR or similar platforms.
• Experience with Airflow, Prefect or similar orchestration tools.
• Git and CI/CD pipelines.
• PyTest and Spark testing approaches.
• Good communication and Agile experience.
• Strong analytical and problem-solving abilities.

Conhecimentos

Python

SQL

Communication

Agile

Problem solving

Performance optimization

Ferramentas

PySpark

Apache Spark

Kafka

Parquet

Delta Lake

Avro

AWS

Azure

GCP

Databricks

EMR

Airflow

Prefect

Git

CI/CD

PyTest

Unit testing

Integration testing

Descrição da oferta de emprego

Wire IT in Portugal seeks a Senior PySpark Developer to join our hybrid team in Porto or Lisbon. You will design and optimize scalable data pipelines using PySpark and Spark, integrate with Kafka, Parquet/Delta Lake, and cloud platforms.

Strong Python, SQL, and testing skills are essential to ensure reliable data processing. You will work in an agile environment, collaborating with cross-functional teams to deliver high-impact solutions and drive performance improvements across streaming and