Lead AWS Data Bricks Spark/Scala Engineer

Highbrow LLC; Remote; EUR 102.000 - 138.000; Vollzeit

Zusammenfassung

A leading tech consulting firm seeks a Lead AWS Data Bricks Spark/Scala Engineer. This role requires over 10 years of experience in architecture and data analytics, and proficiency in AWS and DataBricks. Responsibilities include architecting data solutions and driving team quality ownership. Providing migration expertise from various platforms to AWS is essential. The role supports remote work, making it perfect for skilled professionals looking to make a significant impact while working flexibly.

Qualifikationen

• 10+ years of experience in architecture, design, implementation, and analytics solutions.
• Hands-on development with Databricks.
• Experience with AWS solutioning and Data Migration.

Aufgaben

• Architect data solutions using DataBricks and AWS.
• Drive quality ownership across the team.
• Identify and address code defects to enhance quality.

Kenntnisse

Architecture solutions

Data Engineering

ETL proficiency

AWS Solutioning

Spark/Scala programming

Data Migration

Agile methodology

Tools

DataBricks

AWS

Spark

Jobbeschreibung

Job Title: Lead AWS Data Bricks Spark/Scala Engineer

Job ID: 2022-11882

Job Location: Remote

Job Travel Location(s):

# Positions: 1

Employment Type: W2

Candidate Constraints:

Duration:Long Term

# of Layers:0

Work Eligibility:US Permanent Eligibility to Work Required

Key Technology: DataBricks, AWS, Spark

Job Responsibilities

• Data Bricks Architect profile with AWS combination.
• Data Management, Solutioning, Data Engineering and ETL proficiency using Data Bricks tool / framework which includes programming with Spark/Scala
• Data Management, Migration experience from other platforms to AWS
• Drive mentality of building well architected applications for AWS Cloud
• Drive the mentality of quality being owned by the entire team
• Identify code defects and work with developers to address quality issues in product code.
• Finding bottlenecks and thresholds in existing code through the use of automation tools.
• Articulate clear business objectives aligned to technical specifications and work in an iterative agile pattern daily.
• Ownership over your work task, and are comfortable interacting with all levels of the team and raise challenges when necessary

Skills and Experience Required

Required:

• 10+ years of experience in architecture, design, implementation, and analytics solutions
• Hands on development Design and Develop applications using Databricks
• Experience with Solutioning on AWS
• Data Migration experience from other platforms to AWS
• In depth understanding of Spark Architecture including Spark Core, Spark SQL, Data Frames, Spark Streaming, RDD caching, Spark MLib.

Desired:

• Any programming language experience with SQL, Store procedures, Spark/Scala.
• Strong understanding of Data Modeling and defining conceptual logical and physical data models.
• Ability to design and demonstrate system Architecture with different environments
• Ability to deal with both On-Premises and Cloud systems