Cloud Data Pipeline Reliability Engineer

STEM Search Group; Charlotte (NC); On-site; USD 110,000 - 160,000; Full time

Job summary

STEM Search Group is seeking a Data Engineer to manage cloud data warehouse pipelines and real-time streaming infrastructure. You will optimize Snowflake deployments, tune SQL workloads, and implement automated data quality checks.

You will collaborate with data teams to resolve root causes, write Python tools, and support CI/CD for data infra changes, ensuring reliable, scalable data delivery.

Qualifications

• Hands-on production experience running and tuning a cloud data warehouse (Snowflake or similar).
• Real, working experience with a streaming platform (Kafka or similar): producers, consumers, schema handling, troubleshooting.
• Comfortable in SQL and at least one scripting language, Python preferred.
• Experience with a major cloud provider (AWS, Azure, or GCP).
• Familiarity with orchestration tools like Airflow or dbt.
• Solid grasp of data modeling and the full data lifecycle, from ingestion to delivery.
• Able to explain a pipeline issue clearly to both engineers and non-technical stakeholders.

Responsibilities

• Keep production data pipelines running across a cloud data warehouse and a real-time streaming platform, catching problems before outages.
• Tune warehouse performance, manage access roles, and keep compute costs in check as usage grows.
• Configure and maintain streaming infrastructure (topics, partitions, consumer groups) so data moves in near real time without gaps or duplicates.
• Build automated checks that catch bad data, failed loads, or delivery delays before anyone downstream notices.
• Write scripts and small tools in Python and SQL to cut down manual, repetitive pipeline work.
• Support CI/CD for data infrastructure changes so updates ship safely and roll back cleanly if something breaks.
• Work with data engineering and analytics teams to fix root causes, not just symptoms, when pipelines fail.

Skills

Cloud data warehouse

Streaming platform (Kafka)

SQL

Python

Cloud providers (AWS/Azure/GCP)

Orchestration tools (Airflow/dbt)

Data modeling

Communication with stakeholders

Tools

Snowflake

Kafka

Airflow

dbt

Job description

STEM Search Group is seeking a Data Engineer to manage cloud data warehouse pipelines and real-time streaming infrastructure. You will optimize Snowflake deployments, tune SQL workloads, and implement automated data quality checks.

You will collaborate with data teams to resolve root causes, write Python tools, and support CI/CD for data infra changes, ensuring reliable, scalable data delivery.