Sr. Data Engineer

Sunbelt Rentals, Inc.; Fort Mill (SC); On-site; USD 130,000 - 179,000; Full time

Benefits offered by this job

Health plan

Dental plan

Vision plan

401(k) match

Volunteer time off

Disability insurance

Life insurance

Travel insurance

Flexible spending

Tuition reimbursement

EAP

Length of service awards

Job summary

Sunbelt Rentals, Inc. is seeking a Senior Data Engineer to design, build, and maintain end-to-end IoT data pipelines into Databricks, focusing on streaming ingestion, data quality, and Delta Lake operations.

The role requires strong Python/SQL, Spark, and Kafka experience, with a focus on production stability and collaboration with data scientists and analysts. Onsite in Fort Mill, SC, with a competitive USD salary.

Qualifications

• 5+ years of data engineering experience, including Databricks exposure.
• Strong Python and SQL skills with production-grade Spark code.
• Hands-on with Kafka-based streaming pipelines and Delta Lake.
• Experience in IoT data patterns and cloud infrastructure.

Responsibilities

• Design, build, and maintain end-to-end IoT data pipelines into Databricks.
• Lead streaming ingestion pipelines using Apache Kafka and Databricks Structured Streaming.
• Enforce data engineering best practices, schema governance, and data quality.
• Architect and manage Delta Lake tables, partitioning, and vacuuming.
• Collaborate with infrastructure and cloud teams on Databricks clusters and Unity Catalog.
• Mentor junior engineers and partner with data scientists and analysts.
• Troubleshoot production pipeline failures and drive RCAs.
• Evaluate new tools to improve the data platform.

Skills

Databricks

Python

SQL

Kafka

Spark

IoT data patterns

Cloud infrastructure

Communication skills

Education

Bachelor's degree in Computer Science or related field

Tools

Delta Lake

Delta Live Tables

Unity Catalog

MLflow

Azure

Infrastructure-as-code

Databricks

Job description

Sunbelt Rentals, Inc. is hiring a Sr. Data Engineer to design, build, and maintain end-to-end IoT data pipelines into Databricks. The position focuses on streaming ingestion, data quality, Delta Lake operations, and supporting production stability across the data platform.

Onsite Location

Fort Mill, SC (onsite)

Compensation

USD 130,315 - 179,183 per yearly

Base pay range: $130,315.00 - 179,183.40

Experience Level

Minimum experience: 5 years

Core Responsibilities

• Design, build, and maintain end-to-end IoT data pipelines from edge devices into Databricks using the medallion architecture (Bronze/Silver/Gold).
• Lead development and optimization of streaming ingestion pipelines using Apache Kafka and Databricks Structured Streaming.
• Define and enforce data engineering best practices, including pipeline design patterns, data quality frameworks, and schema governance.
• Architect and manage Delta Lake tables, including partitioning strategies, Z-ordering, and vacuuming to support performance and cost efficiency.
• Collaborate with infrastructure and cloud teams to configure and manage Databricks clusters, Unity Catalog, and Delta Live Tables.
• Mentor junior and mid-level data engineers through code reviews, pair programming, and technical guidance.
• Partner with data scientists and analysts to support downstream analytical and ML requirements.
• Troubleshoot production pipeline failures and drive root cause analysis and resolution.
• Evaluate and introduce new tools and technologies to continuously improve the data platform.

Required Qualifications

• 5+ years of data engineering experience, including at least 2 years working extensively with Databricks.
• Strong proficiency in Python and SQL, with experience writing production-grade Spark code.
• Hands‑on experience building and managing Kafka‑based streaming pipelines.
• Deep familiarity with Delta Lake, Delta Live Tables, and the Databricks ecosystem, including Unity Catalog, Workflows, and MLflow.
• Experience with IoT data patterns such as high‑frequency telemetry ingestion, device state management, and time‑series data.
• Solid understanding of cloud infrastructure ( Azure preferred) and infrastructure‑as‑code concepts.
• Strong communication skills, including the ability to lead technical discussions and translate requirements into architecture.

Technologies

• Databricks
• Apache Kafka
• Databricks Structured Streaming
• Delta Lake
• Delta Live Tables
• Unity Catalog
• Workflows
• MLflow
• Python
• SQL
• Spark
• Azure
• Infrastructure‑as‑code
• Bronze/Silver/Gold (medallion architecture)
• Z‑ordering
• vacuuming
• IoT
• ML

Preferred Qualifications

• Experience with Geotab or similar fleet telematics platforms.
• Databricks Certified Data Engineer Associate or Professional certification.
• Familiarity with Azure Key Vault, Event Hubs, or Azure Data Factory.
• Experience with dbt for data transformation.
• Bachelor’s degree in computer science, Information Technology, Data Engineering, or equivalent experience.

Benefits

• Health, Dental and Vision plans
• 401(k) Match
• Volunteer time off
• Short‑term and long‑term disability
• Accident, Life and Travel insurance, as well as flexible spending
• Tuition Reimbursement Options
• Employee Assistance Program (EAP)
• Length of Service Awards

Benefits Eligibility

You will become eligible for benefits on the first of the month following 30 days from your start date.

Paid Time Off

• 12-25 vacation days depending on years of service
• 5 sick days
• 6 holidays
• 2 half day holidays
• 2 floating holidays
• 1 inclusion day
• 1 volunteer day

Physical Demands and Safety Requirements

• Must be able to bend, squat, crouch and/or reach and lift up to 25 pounds or more, as required by the job.
• Some Sunbelt jobs may require driving for long periods of time, loading and unloading heavy equipment, performing work in extreme weather conditions including rain, wind, or excessive temperatures and/or night and weekend work.
• All duties must be performed according to Sunbelt’s safety policies and guidelines.
• Reasonable accommodations may be made to comply with ADA/ADAAA.