Senior Data Scientist - Azure Databricks & Data Lakehouse

Saipro Systems Llc.; Indianapolis (IN); Hybrid; USD 55,000 - 63,000; Full time

Job summary

Saipro Systems Llc. in Indianapolis, IN seeks a Data Scientist for a hybrid role focused on data integration, pipeline development, governance, and documentation to deliver reliable data products for the agency.

Responsibilities include building batch and streaming pipelines in Azure Databricks with PySpark, Spark SQL, and Delta Lake, designing dimensional data models in Azure Synapse, and applying HIPAA-compliant data governance.

Qualifications

• Master's degree or 4 years of related experience required.
• Bachelor's with 2+ years experience; or 4+ years hands-on work.
• Certifications may substitute for formal education/experience.
• Experience developing with PySpark, Spark SQL, and R is required.
• ACID, MERGE, schema evolution, time travel, and maintenance concepts.
• Medallion architecture (bronze/silver/gold) in data engineering.
• Unity Catalog and workspace governance experience.
• Partitioning, shuffle, caching issues diagnosis in Spark UI.
• ADF Pipelines and Databricks Workflows (Jobs) familiarity.
• Synapse/ADF pipeline copy, data flows, triggers experience.
• Git repos and CI/CD for Synapse and Databricks assets.

Responsibilities

• Assist with data technology strategy and resolve integration issues.
• Implement event-based data integration and universal data vision.
• Oversee design/implementation of data solutions across secure systems.
• Define and enforce data governance policies and standards.
• Provide advice, training, and technical support on projects.
• Gather requirements and explain technical concepts to non-technical staff.
• Collaborate with leads, PMs, architects, data scientists, and analysts.
• Assist data extraction, transformation, and loading from diverse sources.
• Design and maintain batch and streaming pipelines in Azure Databricks.
• Tune data models and SQL workloads in Azure Synapse for reporting.
• Coordinate workflows using Databricks Jobs and Synapse/ADF pipelines.
• Apply security controls for PHI/PII data per HIPAA and policy.

Skills

Databricks PySpark

Spark SQL

Delta Lake

Azure Data Factory

Azure Synapse

Python

SQL

Data governance

Communication

Education

Master's degree in data science or related field

Bachelor's degree + 2+ years experience

Databricks/Azure certifications

Tools

Databricks

Power BI

Job description

Saipro Systems Llc. in Indianapolis, IN seeks a Data Scientist for a hybrid role focused on data integration, pipeline development, governance, and documentation to deliver reliable data products for the agency.

Responsibilities include building batch and streaming pipelines in Azure Databricks with PySpark, Spark SQL, and Delta Lake, designing dimensional data models in Azure Synapse, and applying HIPAA-compliant data governance.