Spark Cloud Data Engineer (21-25k PLN net b2b)

ClearScale; Poland; Remote; PLN 180,000 - 240,000; Full time

Job summary

A cloud services company is looking for a Spark Cloud Data Engineer in Poland to build scalable data solutions for various clients. The role involves analyzing and estimating tasks, designing architectures, and collaborating with teams. Candidates should have AWS experience and a relevant degree, with preferences for Spark and SQL skills. The position offers a fair wage, full-time hours, and the opportunity to work remotely.

Benefits

Fair wage with high hourly rate

Annual rate review

4 weeks of annual vacation

Qualifications

• 1+ years of hands-on AWS experience.
• 3-6 years in Data, Cloud, or Software Engineering roles.
• AWS Machine Learning Specialty certification preferred.

Responsibilities

• Analyze, scope, and estimate tasks.
• Design and implement optimal architecture.
• Collaborate with development and analytical teams.

Skills

AWS hands-on experience

Data Engineering experience

Experience with Amazon EMR or AWS Glue

Hands-on Spark with Java, Scala, or Python

Education

Degree in Computer Science, Statistics, Informatics, or a quantitative field

Tools

Amazon Athena

SQL

Job description

Overview

ClearScale is looking for a Spark Cloud Data Engineer to participate in building cost-efficient scalable data lakes for a wide variety of customers, from small startups to large enterprises.

Typically, projects fall into several categories, including collecting data from IoT edge locations, storing it in the Data Lake, orchestrating ETL processes, and slicing data into various data marts to feed machine learning or BI pipelines; optimizing pre-existing Spark clusters by identifying bottlenecks and implementing improvements; and building data delivery pipelines to ingest high-volume real-time streams, detect anomalies, and store results for dashboards.

Responsibilities

• Analyze, scope, and estimate tasks; identify technology stack and tools
• Design and implement optimal architecture and migration plan
• Develop new and re-architecture solution modules; redesign and refactor program code
• Examine performance and advise on necessary architecture changes
• Communicate with the client on project-related issues
• Collaborate with in-house and external development and analytical teams

Qualifications

ClearScale expects a successful candidate to have most of the following qualifications and skills:

• 1+ years of AWS hands-on experience
• 3-6 years of experience in Data, Cloud, or Software Engineer roles, with a degree in Computer Science, Statistics, Informatics, Information Systems, or another quantitative field
• 1+ years of hands-on experience with Amazon EMR or AWS Glue
• 1+ years of hands-on Spark through SDK: Java, Scala, or Python

Additionally, it would make us smile even more if you have:

• 1+ years of hands-on Amazon Athena experience
• 1+ years of hands-on SQL experience
• AWS certifications, especially AWS Machine Learning Specialty
• Hands-on experience designing efficient architectures for high-load enterprise-scale applications or big data pipelines
• Hands-on experience with general AWS infrastructure components
• Hands-on experience with message queuing, stream processing, and highly scalable big data stores
• Knowledge and experience working with SQL and NoSQL databases
• Proven experience in redesigning and re-architecting large, complex business applications
• Strong self-management and self-organizational skills

Why ClearScale can be a great fit

• You’d like to work remotely with a flexible schedule
• You thrive in a small, dynamic, and agile team that encourages you to learn and grow
• You enjoy finding solutions to interesting problems and figuring out how things work
• You welcome having autonomy with complex tasks
• You are passionate about using your experience and expertise to inspire the team

Benefits

• Fair wage with a high hourly rate in USD
• Full-time, 40 hours per week contract; not a freelance job
• Payments every 2 weeks
• Annual rate review
• Referral program
• 4 weeks of annual vacation

About ClearScale

ClearScale (headquartered in San Francisco, California, USA) is an AWS Premier Consulting Partner and has offered a full range of professional cloud computing services for over 10 years, including architecture design, DevOps automation, refactoring and cloud-native applications development, integration, migration, security, and 24/7 technical support using advanced technologies. We were the third company to gain the AWS Applied AI and Machine Learning Operations (MLOps) competency, a designation held by fewer than 15 partners. Since its foundation, the company has worked 100% remotely from various cities and countries, valuing trust and the professional reputation of its employees.