Senior Data Engineer (AWS and Spark)

Improveit Solutions; Polska; Híbrido; PLN 180.000 - 320.000; Jornada completa

Descripción de la vacante

ImproveIT Solutions is seeking a Senior Data Engineer to design, implement, and optimize scalable data pipelines using AWS, Spark/PySpark, and streaming technologies. You will work on distributed data processing across international projects in a fully remote setup.

The ideal candidate brings 5+ years in data engineering, hands-on AWS expertise, and strong SQL plus NoSQL experience. You will collaborate with cross-functional teams and own technical tasks from requirements through deployment.

Formación

• 5+ years of commercial experience in Data Engineering.
• Strong hands-on experience with AWS.
• Strong experience with Apache Spark / PySpark.
• Experience building and maintaining data pipelines and ETL/ELT processes.
• Hands-on experience with AWS S3.
• Experience with Kafka, Kinesis, or similar streaming technologies.
• Strong SQL skills and experience with PostgreSQL, MySQL, Redshift, or similar databases.
• Experience with DynamoDB or other NoSQL databases.
• Experience working with containerized applications, ideally AWS ECS / Fargate.

Responsabilidades

• Design, develop, and maintain scalable data pipelines and ETL/ELT processes.
• Work with large-scale datasets and distributed data processing systems.
• Build and optimize data processing workflows using Spark / PySpark.
• Develop and maintain data ingestion and streaming solutions using Kafka, Kinesis, or similar technologies.
• Work with AWS data services and cloud-based data infrastructure.
• Analyze and optimize data pipelines for performance, scalability, and reliability.
• Take ownership of technical tasks from requirements analysis through implementation.
• Collaborate with engineers, analysts, and other stakeholders across international projects.
• Contribute to technical decisions and data architecture improvements.
• Work on different international projects depending on your experience and expertise.

Herramientas

Kafka

Kinesis

Amazon EMR

AWS Glue

Apache Flink

Apache Beam

Cassandra/ScyllaDB

Docker

ECS / Fargate

Descripción del empleo

# Senior Data Engineer (AWS and Spark)Join ImproveIT Solutions as a Senior Data Engineer working with AWS, Spark and streaming data across international projects. Remote.Location: RemoteEmployment: Full timeSeniority: SeniorApply for this roleWrite to us directly. We read every application ourselves and reply within a week.We are looking for strong engineers with solid experience in AWS, Spark/PySpark, data pipelines, and distributed data processing who are open to new opportunities and future project engagements.## Responsibilities* Design, develop, and maintain scalable data pipelines and ETL/ELT processes* Work with large-scale datasets and distributed data processing systems* Build and optimize data processing workflows using Spark / PySpark* Develop and maintain data ingestion and streaming solutions using Kafka, Kinesis, or similar technologies* Work with AWS data services and cloud-based data infrastructure* Analyze and optimize data pipelines for performance, scalability, and reliability* Take ownership of technical tasks from requirements analysis through implementation* Collaborate with engineers, analysts, and other stakeholders across international projects* Contribute to technical decisions and data architecture improvements* Work on different international projects depending on your experience and expertise## Nice to have* AWS Glue* Amazon EMR* AWS Batch* AWS Step Functions* Apache Flink* Apache Beam* Cassandra / ScyllaDB* Experience with real-time data processing* Experience with cloud-based data platforms## What we are looking for* 5+ years of commercial experience in Data Engineering* Strong hands-on experience with AWS* Strong experience with Apache Spark / PySpark* Experience building and maintaining data pipelines and ETL/ELT processes* Hands-on experience with AWS S3* Experience with Kafka, Kinesis, or similar streaming technologies* Strong SQL skills and experience with PostgreSQL, MySQL, Redshift, or similar databases* Experience with DynamoDB or other NoSQL databases* Experience working with containerized applications, ideally AWS ECS / Fargate* Strong understanding of distributed data processing and scalable data architectures* Good English communication skills* Strong problem-solving and analytical skills* Ability to work independently and take ownership of technical tasks* Good communication and collaboration skills## Think this is you?Send whatever shows your work best — a CV, a repository, or a paragraph about something you built. The first call is with the engineer you would work with, not a recruiter.