Data Engineer: Large-Scale Pipelines & DW in Azure

American Builders & Contractors Supply Co., Inc.; Beloit (WI); On-site; USD 124,000 - 130,000; Full time

Benefits offered by this job

Health, dental, and vision coverage

401(k) with generous company match

Employee assistance program

Disability insurance

Life insurance

Flex spending

Paid vacation

Paid holidays

Job summary

ABC Supply in Beloit, WI seeks a Senior Data Engineer to design, develop, and deliver large-scale data systems and pipelines. You will collaborate with stakeholders to provide reliable, scalable datasets that support enterprise needs and low total cost of ownership.

Responsibilities include building data models (Kimball), loading data into Snowflake/Dremio, and optimizing ETL/ELT processes using Python, PySpark, Spark SQL, and cloud-native tools on Azure.

Qualifications

• Master’s or Bachelor’s degree with required years of experience.
• Experience with enterprise integration patterns, DW modeling, ETL, and data management.
• Knowledge of cloud-native data engineering projects and scalable pipelines.
• Proficiency in Python, PySpark, Spark SQL and Kimball data models.

Responsibilities

• Design, develop, and deliver large-scale data systems and pipelines.
• Extract, transform, and load data into Azure data lake using ADF, Databricks, PySpark, and SQL.
• Collaborate with stakeholders to provide reliable, scalable enterprise datasets.
• Plan and implement database changes for data warehouse integrations and DW work.

Skills

Enterprise integration patterns

Data warehouse modeling

ETL processes

Database management

Requirements gathering & design

Cloud native modern data engineering

Designing scalable data pipelines

Python

PySpark

Spark SQL

Kimball data models

Education

Master’s degree in Computer Science, IT/IS, or related field

Bachelor’s degree in Computer Science, IT/IS, or related field and 5 years experience

Tools

Python

Pyspark

Spark SQL

Job description

ABC Supply in Beloit, WI seeks a Senior Data Engineer to design, develop, and deliver large-scale data systems and pipelines. You will collaborate with stakeholders to provide reliable, scalable datasets that support enterprise needs and low total cost of ownership.

Responsibilities include building data models (Kimball), loading data into Snowflake/Dremio, and optimizing ETL/ELT processes using Python, PySpark, Spark SQL, and cloud-native tools on Azure.