PySpark Developer

Wire-IT; Lisboa; Híbrido; EUR 60 000 - 80 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Health Insurance

22 days vacation

4 extra days annually (Carnival, X-mas

Meal Allowance on card (Coverflex)

Referral Bonus

Discounts and flexible options

Annual training budget

Team-oriented culture

Growth opportunities

Challenging projects

Resumo da oferta

Wire IT, based in Portugal, is seeking a Senior PySpark Developer to join a senior, agile team. You will design and optimize data pipelines using PySpark and Spark, connecting SQL/NoSQL, Kafka streams, Parquet/Delta Lake, and cloud platforms.

Hybrid work in Porto or Lisbon, Portugal. 5+ years of software/data engineering experience required. Strong Python, SQL and performance tuning skills are essential for this role.

Qualificações

• Minimum 5 years in software development or data engineering.
• Strong experience with PySpark and Apache Spark.
• Advanced Python and SQL skills.
• Experience building scalable batch and streaming data pipelines.
• Deep understanding of Spark internals: DAGs, lazy evaluation, caching, partitioning, shuffling, memory tuning, data skew.
• Experience integrating SQL/NoSQL, APIs, files and Kafka streams.
• Knowledge of Parquet, Delta Lake and Avro.
• Experience with AWS, Azure or GCP.
• Experience with Databricks, EMR or similar platforms.
• Experience with Airflow, Prefect or similar orchestration tools.
• Proficiency with Git and CI/CD.
• Experience with PyTest and testing in Spark environments.
• Good communication and Agile experience.
• Strong analytical and problem-solving abilities.

Conhecimentos

PySpark

Python

SQL

Spark Architecture

Data Pipelines

Kafka Streams

Cloud Platforms

CI/CD

PyTest

Agile

Communication

Performance Tuning

Ferramentas

Databricks

Airflow

Delta Lake

EMR

Descrição da oferta de emprego

Based in Portugal, Wire IT is your specialized IT consulting partner with 18 years of experience driven by an experienced and senior team that helps clients make the right decisions in a fast-moving market.

Wire IT's ambition is to grow while keeping true to its nature: agile, people-centered, and fun. As we like to say: Its not only what we do, its how we manage that truly make the difference.

If you're looking for more than just a job - if you're looking for a place where you can truly make a difference - you might be the next person to join WIRE IT.

Position: Senior PySpark Developer

Location: Porto or Lisbon, Portugal (Hybrid)

Required Skills:

• At least 5 years of experience in software development or data engineering;
• Strong professional experience with PySpark and Apache Spark;
• Advanced proficiency in Python and SQL;
• Experience developing scalable batch and streaming data pipelines;
• Deep understanding of Spark architecture, DAGs, Lazy Evaluation, caching, partitioning, shuffling, memory tuning, and data skew;
• Experience integrating SQL/NoSQL databases, APIs, files, and Kafka streams;
• Knowledge of Parquet, Delta Lake, and Avro;
• Experience with AWS, Azure, or GCP;
• Experience with Databricks, EMR, or similar platforms;
• Experience with Apache Airflow, Prefect, or similar orchestration tools;
• Proficiency with Git and CI/CD pipelines;
• Experience with PyTest, unit testing, integration testing, and Spark data mocking;
• Good communication skills and experience working in Agile environments;
• Strong analytical, problem-solving, and performance-optimization capabilities.

We offer:

• No-term full-time contract;
• Health Insurance;
• 22 days of paid vacation;
• 4 extra days annually (Carnival, Christmas Eve, New Year's Eve, Birthday);
• Meal Allowance on card (Coverflex);
• Referral Bonus;
• Special discounts and flex options;
• Training annual budget;
• A great team-oriented culture;
• Chance to become a part of a dynamically growing international company;
• Challenging projects giving the unique opportunity to grow and learn.

~

Following the EU General Data Protection Regulation No. 2016/679, we inform you that by responding to this announcement, you consent to the use of your data by Wire IT, for as long as necessary. In case of doubt or request for elimination of your personal data, please send an email to [e-mail].