Databricks Data Engineer - SIC & Analytics Specialist

Software Guidance & Assistance, Inc. (SGA, Inc.); New York (NY); On-site; USD 123,984 - 206,640; Full time

Job summary

Software Guidance & Assistance, Inc. (SGA), is seeking a Databricks Data Engineer for a contract assignment with an Insurance Services client in New York, NY. You will design, develop, and implement Databricks-based data pipelines to support ingestion, transformation, and analytics across enterprise data.

The role emphasizes end-to-end data solutions and collaboration with cross-functional teams. Ideal candidates have 7+ years in data engineering with 3+ years hands-on Databricks experience,

Qualifications

• 7+ years in data engineering
• 3+ years with Databricks solutions
• Strong Python/SQL skills
• Experience with Delta Lake, Unity Catalog and Databricks Workflows
• In-depth knowledge of ETL/ELT, data ingestion and modeling
• Familiarity with SIC platforms (e.g., Anaplan, Oracle ICM, Varicent)
• Experience integrating Databricks with cloud platforms and data sources

Responsibilities

• Design and develop Databricks-based data pipelines and dashboards/notebooks
• Maintain and optimize Databricks workspaces and pipelines
• Develop data models and Delta Lake tables
• Support Sales Incentive Compensation processes including quota, attainment, commissions, payouts
• Collaborate with Finance, IT, and BI teams
• Document technical specifications and data architectures
• Provide production support and troubleshoot data issues

Skills

Databricks

Python

SQL

ETL/ELT

Data modeling

Delta Lake

Databricks Workflows

Unity Catalog

Sales Incentive Compensation (SIC)

Cloud platforms (Azure/AWS/GCP)

Problem solving

Tools

Delta Live Tables

Photon

Job description

Software Guidance & Assistance, Inc. (SGA), is seeking a Databricks Data Engineer for a contract assignment with an Insurance Services client in New York, NY. You will design, develop, and implement Databricks-based data pipelines to support ingestion, transformation, and analytics across enterprise data.

The role emphasizes end-to-end data solutions and collaboration with cross-functional teams. Ideal candidates have 7+ years in data engineering with 3+ years hands-on Databricks experience,