Data Scientist

TMS; Indianapolis (IN); On-site; USD 110,000 - 145,000; Full time

Job summary

TMS in Indianapolis seeks a data engineer to develop batch and streaming data pipelines, implement data governance, and collaborate with cross-functional teams to deliver reliable data products for the agency.

You will optimize Azure Databricks and Synapse workloads, design data models and SQL workloads, and contribute to policy, documentation, and mentoring as needed.

Qualifications

• Experience delivering batch and streaming data pipelines.
• Ability to translate business needs into technical designs.
• Knowledge of data governance, PHI/PII protection, and HIPAA considerations.

Responsibilities

• Assists with data technology strategy and data integration issues.
• Designs and implements data solutions for integration across secure systems.
• Defines data management policies and standards for data engineering.
• Provides training and technical support for multiple projects.
• Collaborates with data architects, scientists, analysts and managers on solution design.
• Develops extraction, transformation, and loading of data from diverse sources.
• Builds batch and streaming pipelines in Databricks using PySpark and Delta Lake.
• Loads and tunes dimensional data models and SQL workloads in Synapse.
• Orchestrates data workflows using Databricks and Synapse pipelines; monitors and resumes on failures.
• Applies governance and security controls to protect PHI/PII in line with HIPAA.
• Documents solutions with diagrams, comments, and clear code.

Skills

Batch and streaming data pipelines

Tools

Azure Databricks

PySpark

Spark SQL

Delta Lake

Azure Synapse Analytics

Databricks Workflows

Azure Data Factory

Unity Catalog / data governance

Job description

Local candidates only will be considered for this position.

This role supports data integration, pipeline development, data modeling, governance, and documentation while collaborating with cross-functional teams to deliver high quality, reliable data products for the agency.

Plays a key role in implementing the agency's data strategy by developing batch and streaming pipelines, optimizing data warehouse workloads, and enforcing data governance standards. The position assists with requirements gathering, solution reviews, and technical support, and works closely with data architects, scientists, analysts, and project managers to translate business needs into technical designs. This role supports operational processes, contributes to policy and procedure improvements, and may provide mentorship or supervision based on agency needs.

Job Duties

• Assists with the data technology strategy, including resolvingstrategic data integration issues, such as data quality and stewardship,real-time,
• Event-based data integration, and crafting a universal vision for data integration
• Provides direction and oversight for the design and implementation of data solutions for integration and synthesis across multiple secure systems
• Assists with the definition, implementation, and continued adherence of new data management policies and design standards for data engineering
• Provides advice, training, and technical support for various projects
• Participates in project activities such as requirements gathering, solution reviews, and explaining technical complexities to non-technical personnel
• Work closely with the solution leads, project managers, data architects, data scientists, and data analysts on solution design, architecture, and implementation
• Assists with extraction, transformation, and loading of data from a wide variety of data sources using various data engineering tools and methods.
• Designs, builds, and maintains batch and streaming data pipelines in Azure Databricks using Py Spark, Spark SQL, and Delta Lake, following a medallion (bronze/silver/gold) lake house architecture
• Develops, loads, and tunes dimensional data models and SQL workloads in Azure Synapse Analytics (dedicated and serverless SQL pools) to support reporting and dashboard needs
• Orchestrates and monitors data workflows using Databricks Workflows/Jobs and Synapse or Azure Data Factory pipelines, including scheduling, alerting, and failure recovery
• Applies data governance and security controls (e.g., Unity Catalog permissions, role-based access, row- and column-level security, data masking) to protect sensitive health data (PHI/PII) in accordance with HIPAA and agency policy
• Assists with documenting and demonstrating solutions by developing documentation, flowcharts, layouts, diagrams, charts, code comments and clear code.
• Assists in operations by conducting systems analysis; recommendingchanges in policies and procedures.

The job description is not designed to cover or contain a comprehensive listing of activities, duties or responsibilities that are required of the employee. Other duties, responsibilities and activities may change or be assigned at any time.