AWS Data Migration & Analytics Engineer

RichRelevance; New York (NY, DE); On-site; USD 130,000 - 170,000; Full time

Job summary

RichRelevance seeks a senior Big Data engineer to integrate and optimize complex real-world customer TV and digital ad data for pre- and post-campaign analyses on AWS-based big data platforms. You will work with Databricks, PySpark, Glue, EMR, and Lambda to architect scalable pipelines and collaborate with cross-functional teams.

Requirements include 8+ years in software development, strong experience with Hadoop ecosystem, Scala, Python, and building monitored data workflows on cloud

Qualifications

• 8+ years of software development experience.
• Experience with Hadoop ecosystem and Scala, PySpark, Python.
• Gathers requirements, designs, supports, migrates, automates.
• Monitors data pipelines using PySpark and AWS services.

Responsibilities

• Support, migration, automation and monitoring of data pipelines.
• Ingest data from multiple sources and create aggregated data sets.
• Collaborate with cross-functional teams to meet reporting needs.
• Implement scalable,Cost-effective big data analytics on cloud platforms.

Skills

Big Data Technologies

Scala

PySpark

Python

AWS services

Data pipeline monitoring

Tools

Databricks

Glue

EMR

Step-functions

Lambda

Job description

RichRelevance seeks a senior Big Data engineer to integrate and optimize complex real-world customer TV and digital ad data for pre- and post-campaign analyses on AWS-based big data platforms. You will work with Databricks, PySpark, Glue, EMR, and Lambda to architect scalable pipelines and collaborate with cross-functional teams.

Requirements include 8+ years in software development, strong experience with Hadoop ecosystem, Scala, Python, and building monitored data workflows on cloud