Senior Data Engineer - Databricks & Azure Pipelines

Independence Pet Holdings

Town of Florida (NY)

On-site

USD 120,000 - 180,000

Full time

32 hours ago

Be an early applicant

Application generator

Don’t send a generic resume — generate a resume and cover letter tailored to this exact role.

Get past ATS filters

Benefits offered by this job

Healthcare: comprehensive medical, d5e

Parental Leave

HSA & FSA

Retirement savings plan

Paid time off

Paid holidays & Wellness Day

Job summary

Independence Pet Holdings is seeking a seasoned Senior Data Engineer to build and optimize scalable data pipelines across the medallion architecture. You will ensure data integrity, develop robust data models, and implement governance controls for multiple brands in the North American market.

The ideal candidate has 7+ years of ELT/ETL experience, strong Python/SQL/PySpark skills, and hands-on Databricks with Azure integration.

Qualifications

7+ years of experience building data pipelines using ELT/ETL.
Hands-on Databricks, Spark and Azure data services experience.
Proficient in Python, SQL and PySpark.
Experience with CDC and event/stream-based processing.
Experience with NoSQL and SQL databases.
Strong data modeling, schema design and performance optimization.
Familiar with JSON, Parquet and ORC file formats.

Responsibilities

Design, develop, and deploy robust and scalable data pipelines using Databricks.
Optimize pipeline performance for efficiency and cost-effectiveness.
Contribute to data models in the data lake ensuring integrity and accessibility.
Implement data quality checks and governance compliance.
Integrate Databricks with Azure storage, compute, and security services.