Senior Data Engineer: Build Scalable Data Pipelines

Scale Marketing; Chicago (IL); On-site; USD 110,000 - 150,000; Full time

Job summary

Scale Marketing in Chicago is seeking an experienced Data Engineer to expand and optimize our data pipeline architecture across multiple teams. You will build and maintain scalable data systems, ensure reliable data delivery, and support analysts and data scientists with clean, accessible data.

The right candidate enjoys solving complex data problems and designing end-to-end data solutions for ambitious products.

Qualifications

• Advanced SQL knowledge and experience with relational databases.
• Experience building and optimizing big data pipelines, architectures and data sets.
• Root cause analysis on data and processes to answer business questions.
• Strong analytics with unstructured datasets.
• Data transformation, metadata, dependency and workload management.
• Experience extracting value from large, disconnected datasets.
• Knowledge of message queuing, stream processing, and scalable big data stores.
• Strong project management and organizational skills.
• Experience with cross-functional teams in a dynamic environment.
• 5+ years in a Data Engineer role with a Graduate degree in Computer Science, Statistics, Informatics, Information Systems or similar.
• Experience with big data tools: Hadoop, Hive, Spark, Pig.
• Experience with at least three SQL/NoSQL databases: MongoDB, HBase, Oracle, SQL Server, Cassandra, Redis, Riak.
• Experience with data pipeline/workflow tools: Kafka, Spark, Azure DevOps, AWS Data Pipeline.
• Experience with at least one cloud service: AWS or Azure.
• Experience with at least one stream-processing system: Kinesis, Azure Stream Analytics, Flink.
• Experience with at least three of the following languages: Python, Java, Scala, C++, Linux/Unix

Responsibilities

• Create and maintain optimal data pipeline architecture.
• Assemble large, complex data sets meeting requirements.
• Identify and implement internal process improvements: automating manual processes, optimizing data delivery, redesigning infrastructure.
• Build infrastructure for ETL from various sources using SQL and big data technologies.
• Build analytics tools using the data pipeline for insights into customer acquisition and key metrics.
• Collaborate with Partner, Client and Data teams on data-related issues.
• Keep data separated and secure across national boundaries.
• Create data tools for analytics teams to advance product leadership.
• Work with data and analytics experts to improve data system functionality.

Skills

SQL

Big data pipelines

Data wrangling

Data modeling

Python

Spark

Cloud platforms

ETL processes

Streaming

NoSQL databases

Cross-functional

Education

Graduate degree in CS/Statistics

Tools

Hadoop

Hive

Spark

Pig

MongoDB

Kafka

Azure Data Factory

Job description

Scale Marketing in Chicago is seeking an experienced Data Engineer to expand and optimize our data pipeline architecture across multiple teams. You will build and maintain scalable data systems, ensure reliable data delivery, and support analysts and data scientists with clean, accessible data.

The right candidate enjoys solving complex data problems and designing end-to-end data solutions for ambitious products.