Data Engineer

TechDigital Group; Rosemont (IL); On-site; USD 80,000 - 120,000; Full time

Job summary

An innovative company is seeking a skilled Data Engineer to join their dynamic team. In this role, you will leverage your expertise in AWS, Snowflake, and ETL tools to build and implement efficient data pipelines that meet a wide range of consumer needs. You will be responsible for analyzing new data sources, creating unit tests, and providing application support as part of an on-call rotation. This position offers a unique opportunity to work in an Agile environment, collaborating effectively with all levels of management. If you are passionate about data and thrive in a team-oriented atmosphere, this is the perfect opportunity for you.

Qualifications

• Expertise in Data Warehouse domain and relevant business functions.
• Proven ability to model data for reporting and analytics needs.

Responsibilities

• Build large scale datasets for diverse consumer needs.
• Create and execute unit tests while supporting QA during testing.

Skills

AWS Platform

Snowflake

ETL Tools

Python

SQL

Cloud Technologies

Compute Technologies

Streaming Technologies

API Technologies

CI/CD

Tools

Informatica Power Center

Informatica IICS

Fivetran

Dbt Labs

AWS Airflow

Apache ANT

Maven

Atlassian Bitbucket

Bamboo

Job description

Must Have:

1. AWS Platform
2. Snowflake
3. ETL Tools - Informatica Power center Required; Informatica IICS, Fivetran, Dbt labs, AWS Airflow Nice to have
4. Python, SQL – Required

Familiarity with:

1. Cloud Technologies - AWS services like Lambda, SQS/SNS, EMR, Cloud watch, RDS. EKS
2. Compute Technologies – Spark/Hadoop
3. Streaming Technologies – Kafka, AWS Kinesis
4. API Technologies – MuleSoft, AWS Gateway API
5. CI/CD – Atlassian Bitbucket and Bamboo
6. Build tools - Apache ANT and Maven

RESPONSIBILITIES:

1. Be an expert in Data Warehouse domain and relevant business function.
2. Work with minimal supervision and provide status updates.
3. Provide application support as part of an on-call rotation to resolve outages and user questions.
4. Build large scale datasets for a wide range of consumer needs.
5. Build, test and implement highly efficient data pipelines using a variety of technologies.
6. Analyze new data sources to understand quality, availability, and content.
7. Create and execute unit tests.
8. Support QA team during testing.

Knowledge/Skills/Abilities:

1. Proven ability to model data for reporting and analytics needs.
2. Proven ability to design and implement applications using best practices.
3. Proven ability to analyze and understand existing processes and code.
4. Works and communicates effectively with all levels of management.
5. Excellent written, verbal, and people skills.
6. Must work well within team-oriented environments.
7. Proven ability to work in Agile development methodology.

Mandatory Skills: Snowflake, dbt, ETL, AWS