Data Engineer, Enterprise Data Platforms

Giant Eagle, Inc.; Cranberry Township (Butler County); On-site; USD 110,000 - 150,000; Full time

Job summary

Giant Eagle, Inc. is seeking a Data Engineer on the Enterprise Data Platform team to develop data-centric applications in collaboration with analytics, data science, and business partners.

You will work on data access, integration, modeling and visualization using Databricks, Airflow, PySpark, SQL and Python, with cloud exposure (Azure preferred).

Qualifications

• 2+ years in data engineering with ETL/ELT pipelines at scale.

Responsibilities

• Develop end-to-end complex data projects translating business requirements into data-flow architectures.

Skills

Databricks

Delta Lake

Airflow

Python

SQL

PySpark

Cloud platforms

Medallion architecture

Data modeling

RESTful services

ETL/ELT

Analytics

Education

Bachelor's degree in Computer Science or Mathematics

Advanced degree a plus

Tools

Apache Spark

Tableau

Power BI

Looker

Azure

AWS

Hive

Job description

As a Data Engineer on the Enterprise Data Platform team, you will be working on a team that brings data-centric applications to life. In this role, you will be working with analytics, data science, and business partners to understand capability requirements and develop data solutions based on priorities. This is a developer role in the areas of data techniques including data access, integration, modeling, visualization, mining, design and implementation.

• Databricks – Delta Lake Warehouse concepts
• Airflow – Orchestration and data pipelining
• Pyspark + SQL + Python
• Cloud Platform Exposure – ( Any cloud platform but Azure would be preferred )
• Knowledge on Medallion Architecture
• 2+ years of relevant technical experience working with various data engineering methodologies such as data integration and data pipelines (ETL/ELT) to activate against data at scale.
• 1+years of experience of data modeling for analytic projects activities that include design, curation, and management of large datasets
• Experience with Apache Airflow for orchestrating and scheduling complex data pipelines to ensure reliable and automated ETL workflows.
• 1+years’ experience developing on big data technologies with Spark and Hive, preferably leveraging such as DataBricks, AWS, and Azure equivalent technology.
• Experience leveraging RESTfulweb services to collect and publish data.
• Bachelor's degree in Computer Science, Mathematics, or other technical field or equivalent work experience. Advanced degree an asset

Role Responsibilities:

• Develops and implements end-to-end complex data projects and technical solutions through translating business requirements into technical solutions and data-flow architectures.
• Build and automates data pipelines that clean, transform, and aggregate unorganized data into data sources that are ready for analysis.
• Use expertise to apply various analytic methods to discover and interpret information about data from multiple data sources to implement analytics solutions
• Use expertise in database design to implement, operate stable and scalable dataflows from multiple marketing platforms into a cloud data lake
• Experience with data visualization tools Tableau, PowerBI, and Looker with data modeling
• Design, implement and deploy data applications and mechanisms using big data technology
• Provides subject matter expertise for multiple projects concurrently through all phases of the development lifecycle.
• Develop, enhance, govern, and administer for data platform to: collect data, transform, enrich, unify, segment, and integrate data
• Strong adherence data ethics rules around PII data sets
• Work collaboratively with IT teams, Performance Marketing team, and business leaders to ensure actionable is provided key stakeholders’
• Off hours on-call support rotation as required
• Experience with agile or other rapid application development methods an asset