Senior Databricks Engineer – Remote Data Pipelines

Princeton IT Services, Inc; United States; Remote; USD 130,000 - 180,000; Full time

Job summary

Princeton IT Services, Inc. is seeking a Databricks Engineer to design, develop, and maintain scalable data pipelines using Databricks, PySpark, SQL, and Delta Lake.

You will work on ingestion, transformation, and processing frameworks, with CI/CD automation and secure data governance. The ideal candidate has 6+ years of data engineering experience, 3+ years Databricks hands-on expertise, and a Bachelor's in Engineering.

Qualifications

• Bachelor's degree in Engineering or related field.
• 6+ years in data engineering with 3+ years Databricks hands-on experience.
• Strong Python, PySpark and SQL skills; Delta Lake expertise.

Responsibilities

• Design, develop, and maintain scalable data pipelines with Databricks, PySpark, Python, and SQL.
• Build batch and streaming ingestion frameworks for high-volume data.
• Develop reusable, metadata-driven ingestion and processing frameworks.
• Implement Delta Lake architectures for reliability and performance.
• Configure Unity Catalog access controls and data governance.
• Use Auto Loader for incremental file ingestion and Delta Live Tables/Lakeflow for transforms.
• Implement CI/CD workflows with Azure DevOps and Databricks Asset Bundles.
• Troubleshoot pipeline failures and ensure production reliability.
• Optimize Spark jobs and SQL queries for performance and cost.
• Collaborate with cross-functional teams to deliver reliable data solutions.

Skills

Databricks

PySpark

Python

SQL

Delta Lake

Data Engineering

Unity Catalog

Auto Loader

DLT/Lakeflow

Azure DevOps

Databricks Asset Bundles

Performance Optimization

Education

Bachelor's degree in Engineering

Tools

Azure DevOps

Databricks Asset Bundles

Delta Live Tables (DLT)/Lakeflow

Auto Loader

Job description

Princeton IT Services, Inc. is seeking a Databricks Engineer to design, develop, and maintain scalable data pipelines using Databricks, PySpark, SQL, and Delta Lake.

You will work on ingestion, transformation, and processing frameworks, with CI/CD automation and secure data governance. The ideal candidate has 6+ years of data engineering experience, 3+ years Databricks hands-on expertise, and a Bachelor's in Engineering.