Data Engineer

Scale Marketing

Chicago (IL)

On-site

USD 110,000 - 160,000

Full time

7 days ago

Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Job summary

Scale Marketing is hiring a Data Engineer to expand and optimize its data pipeline architecture, data flow, and ETL processes. The role will build infrastructure for ETL from varied data sources using SQL and big data technologies, and develop analytics tools to drive insights into customer acquisition and operations.

The ideal candidate has 5+ years in a Data Engineer role, advanced SQL skills, and experience with Hadoop, Spark, Hive, Pig, and at least one cloud service (AWS/Azure).

Qualifications

Advanced SQL and relational database experience.
Big data pipelines, architectures, and datasets expertise.
Root-cause analysis on data and processes.
Strong analytical skills with unstructured datasets.
Data transformation, data structures, metadata, dependency/workload management experience.
Proven history with large disconnected datasets.
Knowledge of message queuing, stream processing, and scalable big data stores.
Project management and organizational skills.
Experience with cross-functional teams in a dynamic environment.
5+ years in a Data Engineer role.
Graduate degree in CS/Statistics/Informatics/IS or similar.
Experience with Hadoop/Hive/Spark/Pig.
Experience with 3+ SQL/NoSQL databases.
Experience with data pipeline/workflow tools like Kafka, Spark, Azure DevOps, AWS Data Pipeline.
Experience with at least one cloud: AWS or Azure.
Experience with at least one stream-processing system: Kinesis, Azure Stream Analytics, Flink.
Proficiency in Python, Java, Scala, C++, Linux/Unix.

Responsibilities

Create and maintain optimal data pipeline architecture.
Assemble large, complex data sets to satisfy business requirements.
Automate manual processes and optimize data delivery.
Redesign infrastructure for scalability.
Build ETL infrastructure using SQL and big data technologies.
Develop analytics tools leveraging the data pipeline for insights.
Collaborate with Partner, Client, and Data teams on data issues.
Keep data separated and secure across national boundaries.
Create data tools to support analytics and data scientists.
Work with data and analytics experts to improve data systems.

Skills

Advanced SQL

Root cause analysis

Analytical skills

Cross-functional collaboration

Project management

5+ years Data Engineer experience

Education

Graduate degree in Computer Science or quantitative field

Tools

Apache Hadoop

Apache Hive

Apache Spark

Apache Pig

MongoDB

HBase

Oracle

SQL Server

Cassandra

Neo4j

Redis

Riak

Apache Kafka

Azure DevOps

AWS Data Pipeline

AWS

Microsoft Azure

AWS Kinesis

Azure Stream Analytics

Apache Flink

Python

Java

Scala

C++

Linux/Unix

Job description

Scale Marketing is hiring a Data Engineer to expand and optimize its data pipeline architecture, data flow, and extraction/transform/load processes.

Responsibilities

Create and maintain optimal data pipeline architecture
Assemble large, complex data sets that satisfy functional and non-functional business requirements
Identify, design, and implement internal process improvements, including:
- Automating manual processes
- Optimizing data delivery
- Redesigning infrastructure for greater scalability
Build infrastructure for extraction, transformation, and loading (ETL) from a wide variety of data sources using SQL and big data technologies
Develop analytics tools that leverage the data pipeline to deliver actionable insights into customer acquisition, operational efficiency, and other business performance metrics
Collaborate with stakeholders across Partner, Client, and Data teams to support data-related technical issues and data infrastructure needs
Keep data separated and secure across national boundaries through multiple data centers
Create data tools that support analytics and data scientists in building and optimizing products
Work with data and analytics experts to improve the functionality of data systems

Requirements

Advanced working SQL and experience with relational databases, including:
- Query authoring (SQL)
- Working familiarity with a variety of databases
Experience building and optimizing big data pipelines, architectures, and data sets
Ability to perform root cause analysis on internal and external data and processes to address business questions and identify improvement opportunities
Strong analytical skills with unstructured datasets
Experience building processes supporting:
- Data transformation
- Data structures
- Metadata
- Dependency and workload management
Proven history manipulating, processing, and extracting value from large disconnected datasets
Working knowledge of message queuing, stream processing, and highly scalable big data data stores
Project management and organizational skills
Experience supporting and working with cross-functional teams in a dynamic environment
5+ years in a Data Engineer role
Graduate degree in Computer Science, Statistics, Informatics, Information Systems, or another quantitative field
Experience with big data tools: Apache Hadoop, Apache Hive, Apache Spark, Apache Pig, etc.
Experience with at least three relational SQL and/or NoSQL databases: MongoDB, HBase, Oracle, SQL Server, Cassandra, Neo4j, Redis, and/or Riak
Experience with data pipeline/workflow management tools, including: Apache Kafka and/or Apache Spark, Azure DevOps, AWS Data Pipeline
Experience with at least one cloud service: AWS and/or Microsoft Azure
Experience with at least one stream-processing system: AWS Kinesis, Azure Stream Analytics, and/or Apache Flink, etc.
Experience with at least three object-oriented/object function/operating system scripting languages: Python, Java, Scala, C++, Linux/Unix, etc.