Hybrid Data Engineer: Cloud Pipelines & Data Quality

Agility Partners LLC; Columbus (OH); Hybrid; USD 12,345 - 678,910; Full time

Benefits offered by this job

Medical, Dental, and Vision Insurance

401(k) plan

Life Insurance

Disability coverage

Job summary

A leading utility company in Columbus, OH, is seeking a Data Engineer to work on data pipelines using Azure Data Factory and Databricks. The ideal candidate has a Bachelor's degree and extensive cloud technology experience. This role involves data ingestion, quality testing, and automation in a fast-paced Agile environment, with onsite work 2-3 days per week. Competitive benefits include medical insurance and a 401(k) plan.

Qualifications

• 5+ years of experience in various cloud technologies.
• 6+ years in Enterprise Data Management or Data Engineering role.
• 3+ years building data pipelines using Azure Data Factory, Databricks/Spark.
• 5+ years of data analysis and wrangling experience.

Responsibilities

• Create and maintain the data ingestion and quality testing framework.
• Build and automate data ingestion and aggregation pipelines.
• Set up and improve data quality and audit monitoring.
• Document Data Flow Diagrams and Technical Data Mapping.

Skills

Data ingestion

Data transformation

Azure Data Factory

Databricks/Spark

SQL

Data analysis

Education

Bachelor's degree in Computer Science, Engineering, Mathematics, or a related field

Tools

Snowflake

Kafka

Job description

A leading utility company in Columbus, OH, is seeking a Data Engineer to work on data pipelines using Azure Data Factory and Databricks. The ideal candidate has a Bachelor's degree and extensive cloud technology experience. This role involves data ingestion, quality testing, and automation in a fast-paced Agile environment, with onsite work 2-3 days per week. Competitive benefits include medical insurance and a 401(k) plan.