Remote PySpark & Databricks SQL Consultant (Contract)

Ted Calvert Consulting, LLC; Sacramento (CA); Remote; USD 96,000 - 165,000; Full time

Benefits offered by this job

Remote work

Occasional on-site meetings in SACRAM,

Job summary

Ted Calvert Consulting, LLC seeks an independent PySpark / Databricks SQL consultant for a contract engagement. You will support a State of California department on an Azure Databricks data lake, maintaining ingestion notebooks and resolving job failures.

Ideal candidates have 3–5 years of PySpark/Databricks experience, Unity Catalog, Medallion architecture, and strong Python skills, with PHI/PII data handling expertise. Mostly remote with some on-site Sacramento meetings; 1099/LLC preferred.

Qualifications

• 3-5 years of hands-on PySpark and Databricks development.
• Experience with Databricks Unity Catalog and Medallion architecture.
• Databricks SQL and strong Python skills.
• Experience handling PHI/PII data.

Responsibilities

• Maintain and enhance existing data ingestion notebooks.
• Troubleshoot and resolve failed jobs.
• Participate in code reviews.
• Document code and processes.
• Coordinate with the senior architect and team on priorities, standards, and code reviews.

Skills

PySpark

Databricks SQL

Python

PHI/PII handling

Code reviews

Tools

Azure Databricks

Unity Catalog

Medallion architecture

GitHub workflows

Job description

Ted Calvert Consulting, LLC seeks an independent PySpark / Databricks SQL consultant for a contract engagement. You will support a State of California department on an Azure Databricks data lake, maintaining ingestion notebooks and resolving job failures.

Ideal candidates have 3–5 years of PySpark/Databricks experience, Unity Catalog, Medallion architecture, and strong Python skills, with PHI/PII data handling expertise. Mostly remote with some on-site Sacramento meetings; 1099/LLC preferred.