Spark Cloud Data Engineer (21-25k PLN net b2b)

Pracujw; United States; Remote; USD 80,000 - 100,000; Full time

Job summary

A leading company is seeking a Spark Cloud Data Engineer to build scalable data lakes for various clients. The role requires expertise in AWS and data engineering to design solutions and solve complex problems. This is a remote, full-time position with a competitive hourly rate and opportunities for professional development.

Benefits

High hourly rate

4 weeks of annual vacation

Paid AWS certifications

Flexible schedule

Annual Rate Review

Qualifications

• 1+ years of AWS hands-on experience.
• 3-6 years as Data, Cloud, or Software Engineer.
• 1+ years hands-on Spark through SDK.

Responsibilities

• Design and implement optimal architecture.
• Analyze, scope, and estimate tasks.
• Collaborate with development teams.

Skills

AWS

Data Engineering

Software Engineering

Spark

SQL

NoSQL

Machine Learning

Education

Degree in Computer Science, Statistics, or related field

Tools

AWS Glue

Amazon EMR

Java

Scala

Python

Job description

About the role

ClearScale is looking for a Spark Cloud Data Engineer to participate in building cost-efficient scalable data lakes for a wide variety of customers, from small startups to large enterprises.

Usually, our projects fit into one of the categories (but not limited to them):

Collect data from the IoT edge locations, store it in the Data Lake, orchestrate processes to ETL that data, slice it into the various data marts. Then put those data marts into the machine learning or BI pipelines
Optimize pre-existing Spark clusters, find the bottlenecks in the code and/or architecture, plan and execute improvements
Build a data delivery pipeline to ingest a high volume of the real-time streams, detect anomalies, slice into the window analytics, put those results in the NoSQL for the further dashboard consumption


What will you do on a daily basis:

Analyze, scope, and estimate tasks, identify technology stack and tools
Design and implement optimal architecture and migration plan
Develop new and re-architecture solution modules, re-design and re-factor program code
Examine performance and advise necessary architecture changes
Communicate with the client on project-related issues
Collaborate with in-house and external development and analytical team


Description of candidate requirements

ClearScale expects a successful candidate to have most of the following qualifications and skills::

1+ years of AWS hands-on experience
3-6 years of experience in Data, Cloud, or Software Engineer role, who has attained a degree in Computer Science, Statistics, Informatics, Information Systems, or another quantitative field
1+ years of hands-on of Amazon EMR or AWS Glue
1+ years of hands-on Spark through SDK: Java, Scala, or Python
Additionally, it would make us smile even more if you have:

1+ years of hands-on Amazon Athena experience
1+ years of hands-on SQL experience
AWS certifications, especially AWS Machine Learning Specialty
Hands-on experience designing efficient architectures for high-load enterprise-scale applications or ‘big data’ pipelines
Hands-on experience with general AWS infrastructure components
Hands-on experience with message queuing, stream processing, and highly scalable ‘big data’ stores
Knowledge and experience working with SQL and NoSQL databases
Proven experience in re-design and re-architecting the large complex business applications
Strong self-management and self-organizational skills


ClearScale can be a great fit for you if

You'd like to work remotely with a flexible schedule
You thrive in a small, dynamic, and agile team that encourages you to learn and grow
You enjoy finding solutions to interesting problems and figuring out how things work
You welcome having autonomy with complex tasks
You are passionate about using your experience and expertise to inspire the team


We offer

# 1 Fair wage

High hourly rate in USD
Full-time, 40 hours per week contract. It’s your main and permanent place of work, not a freelance job
Payments every 2 weeks
Annual Rate Review
Referral program
4 weeks of annual vacation


# 2 Independence

From the office location or office conditions. The only requirement - fast and reliable internet.
From big team buzz - we are the community of professional engineers, valuing each others' time and effort
From corruption and bureaucracy - we operate in an honest and competitive environment and we are one of AWS's top 10 key partners.


# 3 Professional Development

Work with innovative Silicon Valley companies and traditional American companies at the cutting edge of digital transformation
We work with the newest technologies in AWS cloud and open-source tools like Jira, Confluence, Lucidchart, Slack, etc
Paid AWS certifications: we provide training material, paid time off, and the examination itself
Horizontal and vertical career growth - We keep growing and people keep growing with us
A nice bonus if you are ready to start at ClearScale quickly


About ClearScale



ClearScale (headquartered in San Francisco, California, USA) - AWS Premier Consulting Partner has been offering a full range of professional cloud computing services for over 10 years, including architecture design, DevOps automation, refactoring and cloud-native applications development, integration, migration, solving all sorts of security issues (from just a security check to preventing cyber-attacks) and 24/7 technical support using the best-advanced technology.

We were the third company to gain a new AWS competence: Applied AI and Machine Learning Operations (MLOps). Less than 15 partners have this competency!

Since the very foundation of the company, we work 100% remotely from various cities and countries. We work on a high trust basis. The professional reputation of our employees is of the highest value.