Data Scientist_W2 Contract

Brilliant Infotech Inc.; Indianapolis (IN); Hybrid; USD 83,000 - 124,000; Full time

Benefits offered by this job

Hybrid work model

12-month contract ROP

Job summary

Brilliant Infotech Inc. is seeking a Data Scientist/Data Engineer with hands-on Azure Databricks, PySpark, Spark SQL and Delta Lake experience to develop and optimize enterprise data pipelines and lakehouse solutions.

The role focuses on data integration, governance, security, and analytics for sensitive agency data in a hybrid Indianapolis setting. Applicants should have strong Azure Databricks and Synapse skills, proficiency in SQL and dimensional modeling, and experience with CI/CD and data

Qualifications

• Hands-on experience with Azure Databricks, PySpark, Spark SQL and Delta Lake.
• Experience with medallion lakehouse architecture, batch/streaming pipelines, and incremental ingestion.
• Strong Azure Synapse Analytics, SQL, dimensional modeling and warehouse optimization.
• Experience with Azure Data Factory, Databricks Workflows and Unity Catalog.
• Knowledge of ADLS Gen2, Entra ID, Key Vault, governance and security controls.
• Experience with Git, CI/CD, Azure DevOps/GitHub, monitoring and troubleshooting.
• Strong communication and technical documentation skills.

Responsibilities

• Develop and optimize enterprise data pipelines and lakehouse solutions.
• Design and tune data warehouse workloads for performance and scalability.
• Support data governance, security and analytics for sensitive agency data.
• Collaborate with stakeholders and document architectures and workflows.

Skills

Azure Databricks

PySpark

Spark SQL

Delta Lake

Lakehouse architecture

Batch/Streaming pipelines

Azure Synapse Analytics

SQL

Dimensional modeling

Data warehousing optimization

Azure Data Factory

Databricks Workflows

Unity Catalog

ADLS Gen2

Microsoft Entra ID

Key Vault

Data governance

Security controls

Git

CI/CD

Azure DevOps

Monitoring

Troubleshooting

Documentation

Communication

Tools

Power BI

Databricks Certification

Job description

Location: Indianapolis, IN (46204) - Hybrid

Type of Contract: W2 Contract

Duration: 12 Months

Required Experience: 04+ Years

Job Description:

Seeking a Data Scientist/Data Engineer with hands-on expertise in Azure Databricks, PySpark, Spark SQL, Delta Lake, and Azure Synapse Analytics to develop and optimize enterprise data pipelines, lakehouse solutions, and data warehouse workloads. The candidate will support data integration, governance, security, and analytics for sensitive agency data.

Required Skills:

• Strong hands-on experience with Azure Databricks, PySpark, Spark SQL, and Delta Lake.
• Experience with medallion lakehouse architecture, batch/streaming pipelines, and incremental ingestion.
• Strong Azure Synapse Analytics, SQL, dimensional modeling, and data warehouse optimization skills.
• Experience with Azure Data Factory, Databricks Workflows, and Unity Catalog.
• Knowledge of ADLS Gen2, Microsoft Entra ID, Key Vault, data governance, and security controls.
• Experience with Git, CI/CD, Azure DevOps/GitHub, monitoring, and troubleshooting.
• Strong communication and technical documentation skills.

Preferred:

• Databricks Certified Data Engineer Associate/Professional.
• Experience with Power BI connectivity and performance optimization.
• Experience protecting sensitive data under HIPAA and agency security policies.