Senior Data Engineer - Cloud Data Platform Lead

Global Partners LP; United States; On-site; USD 182,000 - 190,000; Full time

Benefits offered by this job

Health Insurance

Dental Insurance

Vision Insurance

401k with match

Tuition reimbursement

Job summary

Global Partners LP seeks a senior Data Engineer to architect and operate their production data ingestion, integration, and analytics platform. You will design scalable AWS pipelines with Dagster, develop Snowflake/dbt models, and deploy containerized services on AWS ECS/Fargate.

You will establish CI/CD workflows with GitHub Actions and manage credentials with Secrets Manager, ensuring data quality and privacy." Collaborating with cross-functional teams across wholesale and trading, you’ll lead

Qualifications

• Master’s degree in Computer Science, IT or related field with 5 years in data engineering, software development, or systems architecture.
• Experience designing and supporting integration solutions; proficient in Python for data-intensive apps; REST APIs and Python frameworks like FastAPI; SQL/NoSQL databases.
• 4 years building cloud-native data solutions on AWS/Azure; 4 years with Docker and Kubernetes.
• 2 years building pipelines with ETL/ELT tools (Fivetran, Dagster, dbt, Snowflake); CI/CD with GitHub Actions; IaC with CloudFormation/Terraform; observability and data privacy/compliance.

Responsibilities

• Architect, build, and operate production data ingestion, integration, analytics, and data-platform solutions.
• Design scalable pipelines in AWS with Dagster; develop analytics models in Snowflake/dbt; deploy containerized services on AWS ECS/Fargate using Docker and CloudFormation.
• Establish CI/CD workflows via GitHub Actions; manage credentials with AWS Secrets Manager; implement monitoring, observability, and data-quality controls.
• Build and support source-ingestion, API, reverse-ETL, and backend data apps using Fivetran, Python, FastAPI, and Streamlit.
• Develop master data management solutions and automated data-governance guardrails for key entities.
• Apply AI-assisted development tools and build scalable data products using modern agile practices.
• Partner with terminal, wholesale, retail, supply, and trading teams; mentor and maintain runbooks; promote automation-first engineering.

Skills

Python

REST APIs

FastAPI

SQL

NoSQL

AWS

Docker

Kubernetes

Dagster

dbt

Snowflake

Fivetran

GitHub Actions

CloudFormation

Terraform

Prometheus

Grafana

Metaplane

Streamlit

Education

Master’s degree in Computer Science / IT

Tools

Docker

Kubernetes

AWS

Azure

CloudFormation

Terraform

GitHub Actions

Snowflake

Fivetran

Dagster

dbt

FastAPI

Job description

Global Partners LP seeks a senior Data Engineer to architect and operate their production data ingestion, integration, and analytics platform. You will design scalable AWS pipelines with Dagster, develop Snowflake/dbt models, and deploy containerized services on AWS ECS/Fargate.

You will establish CI/CD workflows with GitHub Actions and manage credentials with Secrets Manager, ensuring data quality and privacy." Collaborating with cross-functional teams across wholesale and trading, you’ll lead