Senior Data & Optimization Engineer – Big Data & Forecasting

Roberts Recruiting, LLC; Boston (MA); On-site; USD 120,000 - 150,000; Full time

Benefits offered by this job

Medical benefits

401k matching

Paid Parental Leave

Discretionary Time Off

Employee Stock Purchase Plan

Job summary

A technology firm in Boston is seeking a Senior Software Engineer with a passion for building highly-scalable, distributed Big Data applications. You will design and implement features for a Forecasting and Optimization platform, collaborating with Data Scientists. Ideal candidates have a minimum of 5 years experience and familiarity with technologies like Java/Scala, Spark, and Kafka. This role offers competitive benefits alongside a collaborative work environment.

Qualifications

• Minimum 5 years of software engineering experience.
• Experience with large scale distributed systems and big data processing.
• Familiarity with real-time data streaming technologies.

Responsibilities

• Hands-on programming capabilities in Java/Scala, Spark, Hadoop.
• Work with Data Scientists in revamping systems.
• Design/architect and implement new features.

Skills

Java/Scala

Big Data processing

Distributed systems

Kafka

Spark

Hadoop

Education

BS or MS in a related field or equivalent experience

Tools

GIT

Kafka

Spark

Hadoop

Solr

Cassandra

Job description

We are looking for a Senior Software Engineer with areal passion forbuilding highly-scalable, distributed Big Data applications based on the Lambda Architecture.We want people who take ownership of a project.As part ofengineering,you’ll work on an agile teamdesigningandbuildinga Forecasting and Optimization platform capable of:

• Processing 2 Billion Bid requests a day
• 1.5TB of data processed daily
• Running in a distributed environment with 500+ servers acrossa hybrid cloud environment
• Data processing in AWS
• 200k QPS at low latency

The optimizer and forecaster take in mathematical models in order to make as well as predict the outcome of "fuzzy" decisions related to inventory management, bidding, and performance of advertising campaigns.

Tools We Like You To Have:

• Scala orJava
• Kafka, Zookeeper,Spark, Hadoop,EMR
• Ansible/Puppet, DataDog

What You'll Be Doing..

• Hands-on programming capabilities in Java/Scala,Spark, Hadoopetc..
• You will Work with a team ofData Scientists in revamping ourForecaster and Optimization systemsfeeding directly into the bidder
• Design/architect and implement new features and systems with a focus on high quality well tested code
• Collaboratewith otherteam members, participating in architecture discussions and code reviews.
• You will provide production support, monitoring the performance of ourForecaster and Optimization systemsand troubleshoot/fix issues that arise with deployed software

You...

• Minimum 5 years software engineering experience
• Experience with large scale distributed systems and big data processing
• Familiarity withreal time data streaming technologies like Spark, Kafka etc
• Experience with GIT version control

Compensation and Benefits:

Wexf place an emphasis and importance on ensuring our total rewards arecompetitive,aligned with industry and to help youcreatea productive work/life balance. Benefits are highly subsidized and include medical, company paid dental, vision, employercontributedHealth Savings Account, 401k matching, corporate gym discounts, pre-tax health and commuter savings, life insurance, 5 and 10-year Sabbatical programs, Discretionary Time Off (a.k.a. open vacation policy!), Paid Parental Leave, an Employee Referral Program, Employee Stock Purchase Plan (ESPP), and much more! All this is withina collaborativework environment you can personalize and topped with engaging programs like Micro-Mentorship, and Team Sports.