Data Engineer - Qlik Replicate/Snowflake

Technology Partners; St. Louis (MO); On-site; USD 110,000 - 165,000; Full time

Job summary

Technology Partners is seeking a hands-on Data Engineer to design, test, and maintain data pipelines for analytics. You will transform complex datasets and ensure alignment with strategic goals while collaborating with IT and business experts.

You must have 5+ years in data warehousing/analytics, strong Snowflake production experience, dbt, SQL, and CDC/replication tools like Qlik Replicate or similar. Experience with Python is required.

Qualifications

• Bachelor's degree in applicable field.
• 5+ years Data warehousing/analytics and data engineering.
• CDC Change Data Capture / Data Ingestion - Experience with Qlik Replicate or similar tools.
• Snowflake - 3+ years production experience with large datasets and performance.
• dbt - hands-on production experience building and maintaining dbt models, tests, deployments.
• SQL - 5+ years advanced development, troubleshooting, and optimization.
• Production Support - able to troubleshoot pipeline failures and data quality issues.
• 2+ years Python in OO style.

Responsibilities

• Automate data wrangling from diverse sources (files, databases, APIs, queues).
• Identify opportunities to improve system and business process efficiencies.
• Cleanse/transform data for analytics.
• Leverage scripting and programming languages as needed.
• Assist in deploying and maintaining ML/AI data pipelines.
• Collaborate with vendors/consultants as appropriate.
• Design scalable data models handling SCDs and performance tuning.
• Apply lean thinking to reduce waste in the role.

Skills

Data warehousing

SQL

Python

Snowflake

dbt

CDC/Replication

Qlik Replicate

Fivetran

HVR

Informatica

Debezium

Git

Dagster

Education

Bachelor's degree

Tools

Qlik Replicate

Fivetran

HVR

Informatica

IBM CDC

Striim

Debezium

Job description

As a Data Engineer, you will design, test, and maintain data pipelines for advanced analytics. You'll transform complex datasets, ensuring alignment with strategic goals, and collaborate with IT and business experts to deliver crucial insights in adherence to information best practices.

WHAT YOU'LL DO

• Automate data wrangling from a wide variety of sources (flat files, remote servers, databases, legacy systems, message queues, APIs and more)
• Identify opportunities to improve system and business process efficiencies
• Cleanse/transform data in preparation for all levels of analytical use
• Leverage many technologies and methodologies as required- includes scripting and programming languages
• Assist in the preparation, deployment, and maintenance of machine learning/artificial intelligence data pipelines
• Work with vendors and consultants as appropriate.
• Assist with designing scalable data models for real-world applications, including handling slowly changing dimensions (SCDs) and performance tuning.
• Applies lean thinking and tools to identify and eliminate waste in all areas of the position

CANDIDATE PROFILE

Target experience: 5+ years in data engineering with strong recent experience in modern cloud data platforms.

We want a hands-on Data Engineer, not primarily a BI developer or architect.

The strongest candidate will have experience across:

Direct Qlik experience is preferred but not required. Strong experience with another enterprise CDC/replication platform should transfer well.

YOU MUST HAVE

• Bachelor's degree in applicable field
• 5+ years Data warehousing/analytics, and data engineering
• CDC Change Data Capture / Data Ingestion - Experience with Qlik Replicate Strong: (may also have similar experience with Fivetran, HVR, Informatica, IBM CDC, Striim, Debezium.)
• Snowflake - Strong: 3+ years Production experience with Snowflake, large datasets, performance, and troubleshooting.
• dbt - Strong: Hands-on production experience building and maintaining dbt models, tests, incremental processing, and deployments.
• SQL - Advanced: 5+ years - Strong SQL development, troubleshooting, optimization, and data warehousing skills.
• Production Support: Able to independently troubleshoot pipeline failures, data quality issues, CDC latency, and other production problems.
• 2+ years of experience in object-oriented programming using Python
• Strong interpersonal skills with professional verbal and written communication abilities

Preferred / Nice to Have

• Dagster or similar orchestration experience
• Git / CI/CD
• DataStage
• Db2 and/or SQL Server

YOU MAY ALSO HAVE

• Bachelor's degree in Business Analytics, Computer Science, Mathematics, Statistics, or related field
• Ability to work independently as well as within a team environment
• Effective project management and planning skills