Senior Data Engineer

Piper Companies; United States; Remote; USD 140,000 - 160,000; Full time

Job summary

A leading shipping and logistics organization is looking for a Senior Data Engineer to manage their data ecosystem. The role involves developing solutions using Azure technologies, creating real-time pipelines, and collaborating with internal teams to meet data needs. The ideal candidate will have extensive experience in data architecture and strong skills in Azure Data Factory and Databricks.

Benefits

Medical

Dental

Vision

401K

PTO

Sick Leave

Holidays

Qualifications

• 7+ years of experience in data strategies, analysis, architecture, and management.
• Experience in building ETL/data warehouse transformation processes.

Responsibilities

• Work with Azure Databricks to develop and deploy solutions for reporting and analytics.
• Create real-time streaming pipelines.
• Data movement and orchestration using Azure Data Factory.

Skills

Azure Data Factory

Databricks

SQL

Python

Scala

Tools

Apache Kafka

Azure Storage

Azure SQL Data Warehouse

Azure Data Lake

Azure Cosmos DB

Azure Stream Analytics

Job description

Piper Companies is seeking a Senior Data Engineer to join a leading shipping and logistics organization. The Senior Data Engineer will handle operational aspects of the data ecosystem, including monitoring, automation, and data flow.

Responsibilities of the Senior Data Engineer include:

• Work with Azure Databricks to develop and deploy solutions for reporting and analytics.
• Create real-time streaming pipelines.
• Data movement and orchestration using Azure Data Factory.
• Work with internal customers and partners to gather requirements, build, and deploy robust models.
• Develop code to aggregate and transform data to ADLS curated zone.
• Provide ongoing support of Databricks code.

Qualifications for the Senior Data Engineer include:

• 7+ years of experience in data strategies, analysis, architecture, and management
• Strong skills in Azure Data Factory, Databricks, SQL, Python (PySpark) and/or Scala
• Experience in building ETL/data warehouse transformation processes
• Hands-on experience with Azure Data Analytics platform (Azure Storage, Azure SQL Data Warehouse, Azure Data Lake, Azure Cosmos DB, Azure Stream Analytics)
• Experience with Apache Kafka and event-based data

Compensation for the Senior Data Engineer include:

• Salary: $140,000 - $160,000 -- Hourly: $73/hr - $77/hr
• Comprehensive Benefits: Medical, Dental, Vision, 401K, PTO, Sick Leave as required by law, and Holidays

This job opens for applications onApril 21, 2025. Applications for this job will be accepted for at least 30 days from the posting date.

#LI-JT2 #LI-REMOTE

Keywords: Data Architecture, ETL, Data Warehousing, Big Data, SQL, Python, Spark, Hadoop, Data Pipelines, Cloud Computing, AWS, Azure, GCP, Data Modeling, Data Integration, Machine Learning, Data Governance, Data Security, NoSQL, Real-time Data Processing