Spark Big Data Engineer - Pipelines, Scala & Cloud Storage

Dexvion

Alpharetta (GA)

On-site

USD 100,000 - 160,000

Full time

33 hours ago
Be an early applicant

Get a reply from this employer

Your resume and cover letter, tailored to exactly what they’re hiring for.

Get past ATS filters

Job summary

Dexvion in Alpharetta, GA, is seeking a Big Data Engineer to transform data for data science and product teams. You will work with Linux, Hadoop, Scoop, and Spark (Scala) using DataFrames and Spark SQL for fast processing.

Responsibilities include building ETL pipelines in Azure Data Factory, integrating Azure and AWS storage, and developing a Spark streaming pipeline in Java. Some travel/relocation may be required.

Qualifications

  • Master's degree in a closely related field is expected.
  • Experience with Spark using Scala and DataFrames for processing data.
  • Knowledge of Hadoop ecosystem, Linux systems, and data ingestion pipelines.

Responsibilities

  • Transform data for data science and product teams using Spark, Scala, and SQL.
  • Create Spark streaming and batch pipelines; leverage ADF for data integration across Azure and AWS services.

Skills

Spark (Scala)
Hadoop
SQL
DataFrames
Spark SQL
Java
Linux

Education

Master's degree in Computer Science/IT/IS/Engineering

Tools

Scoop
Azure Data Factory
Azure Blob Storage
Azure SQL Data Warehouse
AWS S3

Job description

Dexvion in Alpharetta, GA, is seeking a Big Data Engineer to transform data for data science and product teams. You will work with Linux, Hadoop, Scoop, and Spark (Scala) using DataFrames and Spark SQL for fast processing.

Responsibilities include building ETL pipelines in Azure Data Factory, integrating Azure and AWS storage, and developing a Spark streaming pipeline in Java. Some travel/relocation may be required.

Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Senior Azure Data Engineer: Pipelines, Databricks & Big Data
Senior Azure Data Engineer: Pipelines, Databricks & Big Data

CloudBigD • Houston (TX)

On-site
USD 120,000 - 160,000
Data Engineer: Spark, Hadoop & ETL Pipelines
Data Engineer: Spark, Hadoop & ETL Pipelines

Quantum Technologies. LLC • Alpharetta (GA)

On-site
USD 120,000 - 180,000
Senior Data Engineer: ETL Pipelines, Spark & Cloud
Senior Data Engineer: ETL Pipelines, Spark & Cloud

CloudBigD • Mount Laurel Township (NJ)

On-site
USD 81,000 - 115,000
Senior Data Engineer: Build Scalable Pipelines
Senior Data Engineer: Build Scalable Pipelines

Dexian • Charlotte (NC)

On-site
USD 100,000 - 140,000
Databricks Data Engineer - Azure, CDC Pipelines, PySpark
Databricks Data Engineer - Azure, CDC Pipelines, PySpark

CloudBigD • Phoenix (AZ)

On-site
USD 81,000 - 115,000
Senior Azure Databricks Data Engineer: Delta Lake Pipelines
Senior Azure Databricks Data Engineer: Delta Lake Pipelines

EXL • Detroit (MI)

On-site
USD 120,000 - 190,000
Senior Azure Data Engineer: Pipelines, Databricks & SQL
Senior Azure Data Engineer: Pipelines, Databricks & SQL

Valiant Technologies LLC • Houston (TX)

On-site
USD 120,000 - 170,000
Senior Snowflake Data Engineer | Cloud Data Pipelines & AWS
Senior Snowflake Data Engineer | Cloud Data Pipelines & AWS

Dexian • Coppell (TX)

Hybrid
USD 100,000 - 130,000
Data Engineer - Databricks + ADF Data Pipelines
Data Engineer - Databricks + ADF Data Pipelines

Ethereum Technologies LLC • Plano (TX)

On-site
USD 118,000 - 136,000
Big Data Engineer - Databricks & Data Pipelines
Big Data Engineer - Databricks & Data Pipelines

Agilisium Onshore • Westlake Village (CA)

On-site
USD 140,000 - 210,000