Senior Data Engineer - Scalable Forecasting Pipelines

Compunnel, Inc.; Waltham (MA); On-site; USD 110,000 - 140,000; Full time

Job summary

A tech company is seeking a Senior Data Engineer to join their cross-functional team. The role involves designing scalable data pipelines and modernizing architecture. Candidates need a Bachelor's degree and a minimum of 5 years of experience in data engineering, with strong skills in Python, PySpark, SQL, and cloud technologies like Azure. This position offers the opportunity to work with innovative data solutions and the chance to mentor junior engineers.

Qualifications

• Minimum 5 years of experience in analytics, data engineering, or business intelligence.
• Hands-on experience with Databricks.
• Experience developing integrations across multiple systems and APIs.

Responsibilities

• Develop, test, document, and support scalable data pipelines.
• Build and evolve data integrations, including APIs.
• Establish and follow data governance processes to ensure data availability, consistency, integrity, and security.

Skills

Data engineering

API integration

Cloud technologies

Python

PySpark

SQL

Education

Bachelor’s degree in Computer Science, Mathematics, Statistics, or related field

Tools

Databricks

Azure Data Lake

ADF

Azure DevOps

Azure Functions

Snowflake

Job description

A tech company is seeking a Senior Data Engineer to join their cross-functional team. The role involves designing scalable data pipelines and modernizing architecture. Candidates need a Bachelor's degree and a minimum of 5 years of experience in data engineering, with strong skills in Python, PySpark, SQL, and cloud technologies like Azure. This position offers the opportunity to work with innovative data solutions and the chance to mentor junior engineers.