Senior Data Engineer: Scale Data Pipelines & Streaming

Sonoma Consulting Inc.; Chicago (IL); On-site; USD 90,000 - 130,000; Full time

Job summary

A premier IT consulting company in Chicago is looking for a Senior Data Engineer to empower the company with data. This role involves selecting and building solutions using tools like Apache Kafka and providing best practices for data pipelines. The ideal candidate will have extensive programming experience in Java, Python, and SQL, alongside a background in large-scale distributed infrastructures and cloud technologies. This is a full-time role, encouraged for U.S. citizens and authorized workers.

Qualifications

• 2+ years of experience in handling large datasets working with a messaging system such as Apache Kafka or Rabbit MQ.
• 5+ years of high availability, concurrent, large-scale distributed infrastructures experience.
• 7+ years of strong programming expertise in Java, Python, SQL.
• 3+ years working in a highly collaborative environment.
• Expertise in leading cloud technologies like Amazon Web Services.

Responsibilities

• Select, build, and configure solutions using open source platforms like Kafka and Nifi.
• Help build and optimize data pipelines for understanding user base.
• Implement data pipelines to automate ingestion and transformation of data.
• Work with Business Analysts and Data Scientists using agile methodologies.

Skills

Apache Kafka

Rabbit MQ

NoSQL datastores

Java

Python

SQL

Amazon Web Services

Education

Bachelor’s in Computer Science, Computer Engineering or comparable experience

Job description

A premier IT consulting company in Chicago is looking for a Senior Data Engineer to empower the company with data. This role involves selecting and building solutions using tools like Apache Kafka and providing best practices for data pipelines. The ideal candidate will have extensive programming experience in Java, Python, and SQL, alongside a background in large-scale distributed infrastructures and cloud technologies. This is a full-time role, encouraged for U.S. citizens and authorized workers.