Pyspark Developer

Wire IT; Lisboa; Híbrido; EUR 65 000 - 90 000; Tempo integral

Vantagens oferecidas por esta oferta de emprego

Health Insurance

Vacation 22 days

4 extra days annually

Meal Allowance (Coverflex)

Referral Bonus

Discounts and flex options

Training annual budget

Team-oriented culture

Growth with international company

Challenging projects

Resumo da oferta

Wire IT in Portugal seeks a Senior PySpark Developer to join our hybrid team in Porto or Lisbon. You will design and optimize scalable data pipelines using PySpark and Spark, integrate with Kafka, Parquet/Delta Lake, and cloud platforms.

Strong Python, SQL, and testing skills are essential to ensure reliable data processing. You will work in an agile environment, collaborating with cross-functional teams to deliver high-impact solutions and drive performance improvements across streaming and

Qualificações

• 5+ years of software development or data engineering experience.
• Strong PySpark and Apache Spark experience.
• Advanced Python and SQL skills.
• Experience building scalable batch and streaming data pipelines.
• Deep understanding of Spark architecture, DAGs, and memory tuning.
• Experience with SQL/NoSQL databases, APIs, files, and Kafka streams.
• Knowledge of Parquet, Delta Lake, and Avro.
• Experience with cloud platforms (AWS/Azure/GCP).
• Experience with Databricks, EMR or similar platforms.
• Experience with Airflow, Prefect or similar orchestration tools.
• Git and CI/CD pipelines.
• PyTest and Spark testing approaches.
• Good communication and Agile experience.
• Strong analytical and problem-solving abilities.

Conhecimentos

Python

SQL

Communication

Agile

Problem solving

Performance optimization

Ferramentas

PySpark

Apache Spark

Kafka

Parquet

Delta Lake

Avro

AWS

Azure

GCP

Databricks

EMR

Airflow

Prefect

Git

CI/CD

PyTest

Unit testing

Integration testing

Descrição da oferta de emprego

Based in Portugal, Wire IT is your specialized IT consulting partner with 18 years of experience driven by an experienced and senior team that helps clients make the right decisions in a fast-moving market.

Wire IT's ambition is to grow while keeping true to its nature: agile, people-centered, and fun.

As we like to say: Its not only what we do, its how we manage that truly make the difference.

If you're looking for more than just a job - if you're looking for a place where you can truly make a difference - you might be the next person to join WIRE IT.

Position

Senior PySpark Developer

Location

Porto or Lisbon, Portugal (Hybrid)

Required Skills

• At least 5 years of experience in software development or data engineering;
• Strong professional experience with PySpark and Apache Spark;
• Advanced proficiency in Python and SQL;
• Experience developing scalable batch and streaming data pipelines;
• Deep understanding of Spark architecture, DAGs, Lazy Evaluation, caching, partitioning, shuffling, memory tuning, and data skew;
• Experience integrating SQL/NoSQL databases, APIs, files, and Kafka streams;
• Knowledge of Parquet, Delta Lake, and Avro;
• Experience with AWS, Azure, or GCP;
• Experience with Databricks, EMR, or similar platforms;
• Experience with Apache Airflow, Prefect, or similar orchestration tools;
• Proficiency with Git and CI/CD pipelines;
• Experience with PyTest, unit testing, integration testing, and Spark data mocking;
• Good communication skills and experience working in Agile environments;
• Strong analytical, problem-solving, and performance-optimization capabilities.

We Offer

• No-term full-time contract;
• Health Insurance;
• 22 days of paid vacation;
• 4 extra days annually (Carnival, Christmas Eve, New Year's Eve, Birthday);
• Meal Allowance oncard (Coverflex);
• Referral Bonus;
• Special discounts and flex options;
• Training annual budget;
• A great team-oriented culture;
• Chance to become a part of a dynamically growing international company;
• Challenging projects giving the unique opportunity to grow and learn.

~

Following the EU General Data Protection Regulation No. ********, we inform you that by responding to this announcement, you consent to the use of your data by Wire IT, for as long as necessary.

In case of doubt or request for elimination of your personal data, please send an email to ******.