Senior Data Engineer - Contract-to-Hire | ETL, Kafka, PySpark

Brooksource; Kansas City (MO); On-site; USD 76,000 - 83,000; Full time

Benefits offered by this job

Dental insurance

Health insurance

Vision insurance

401k with company match

Paid time off

EAP

Job summary

Brooksource is hiring a Senior Data Engineer for a 12-month contract-to-hire role based on-site in Kansas City, MO. You will design and build reliable data pipelines turning raw inputs into clean datasets for enterprise analytics and AI/ML consumption.

This role emphasizes hands-on ETL/ELT work, data modeling, and maintaining production pipelines through monitoring, validation, and governance practices.

Qualifications

• Bachelor degree in CS or related field with substantial professional experience.
• Associates degree with substantial experience (10+ years).
• High School Diploma/GED with 12+ years of related experience.
• Strong Python and PySpark for large-scale data processing.
• Proficiency in SQL for data manipulation and tuning.
• Experience with Dataform, Dataproc, BigQuery for pipelines.
• Knowledge of data governance principles and implementations.
• Experience using Git and CI/CD for deployments.
• Experience building/restful APIs and enterprise integrations.

Responsibilities

• Design, develop, and maintain scalable data pipelines for analytics and AI/ML consumption.
• Collaborate with cross-functional teams to translate data requirements into engineering solutions.
• Build ingestion frameworks using Kafka Confluent with multiple data sources.
• Develop ELT/ETL pipelines using PySpark, SQL, Dataform, DBT, and Dataproc.
• Model data structures and warehousing for performance and scalability.
• Ensure data quality, integrity, and governance with monitoring.
• Implement data lineage, metadata management, privacy, and compliance.
• Develop automated monitoring and alerting for reliability and performance.
• Support downstream reporting, analytics, and AI/ML model consumption.
• Follow CI/CD best practices using Git and IaC tools.
• Provide operational support and troubleshoot pipelines in production.
• Optimize data processing and resource utilization in Azure environments.
• Build RESTful API integrations for ingestion and sync with apps.

Skills

Python

PySpark

SQL

ETL/ELT

RESTful APIs

Git & CI/CD

Data governance

BigQuery

Dataproc

Kafka

Confluent

Education

Bachelor's degree in CS or related field

Associate degree

High School Diploma/GED

Tools

Dataform

DBT

Dataproc

BigQuery

Kafka

Confluent

Azure

Git

Job description

Brooksource is hiring a Senior Data Engineer for a 12-month contract-to-hire role based on-site in Kansas City, MO. You will design and build reliable data pipelines turning raw inputs into clean datasets for enterprise analytics and AI/ML consumption.

This role emphasizes hands-on ETL/ELT work, data modeling, and maintaining production pipelines through monitoring, validation, and governance practices.