Data Scientist in Indianapolis

Energy Jobline ZR; Indianapolis (IN); Hybrid; USD 110,000 - 160,000; Full time

Job summary

Energy Jobline ZR is seeking a Data Scientist in Indianapolis, IN, with a strong background in Databricks, PySpark, Spark SQL, and R. The role focuses on building and maintaining data pipelines and lakehouse architectures in a modern cloud data platform.

The ideal candidate will work across Databricks and Azure technologies, collaborate with analytics teams, and contribute to scalable data modeling and BI-ready data marts. Indianapolis office presence with flexible remote days is offered.

Qualifications

• Experience developing Databricks notebooks using PySpark, Spark SQL and R.
• Experience with Delta Lake/Parquet in ADLS Gen2 and data lakehouse patterns.
• Familiarity with Unity Catalog, data governance and lakehouse architectures.
• Strong SQL, data modeling, and star-schema design.

Skills

Databricks notebooks

PySpark

Spark SQL

R programming

Delta Lake

ADLS Gen2

Unity Catalog

Data modeling

SQL

Git workflows

Education

Master's degree in data science or related field

Bachelor's degree in related field + 2+ years experience OR 4+ years hands-on data engineering

Certifications/substitutions may replace formal education/experience

Tools

Databricks

Azure Synapse

Auto Loader

DLT

A DF Pipelines

GitHub

Azure DevOps

PolyBase

Job description

Job Description

Data Scientist

Indianapolis, IN (One day in office 4 days remote)

• Master's degree in data science, analytics, informatics, computer science, mathematics, engineering, or related field; ...or
• Bachelor's degree in a related field plus 2+ years of relevant experience; or 4+ years of hands‑on experience in a similar data engineering role; OR..
• Relevant certifications and coursework may substitute for formal education/experience requirements
• Developing with Databricks notebooks in PySpark, Spark SQL, and R, including transformations, joins, aggregations, and window functions
• ACID transactions, MERGE upserts, schema enforcement/evolution, time travel, and table maintenance (OPTIMIZE, Z‑order, VACUUM)
• Working with medallion (bronze/silver/gold) architecture and incremental ingestion via Auto Loader, Structured Streaming, or Lakeflow/DLT
• Using Unity Catalog for Workspace Catalogs, schemas, permissions, lineage, external locations, and storage credentials
• Right-sizing clusters and SQL warehouses and diagnose partitioning, shuffle, skew, and caching issues via Spark UI
• Scheduling and managing dependencies with ADF Pipelines, Databricks Workflows (Jobs), and using Git folders (Repos) for source control
• Designing MPP warehouse tables, including distribution (hash, round‑robin, replicated), column store indexes, and partitioning
• Querying Parquet, Delta, and CSV files in ADLS Gen2 using OPENROWSET, external tables, and views
• Building star‑schema facts, dimensions, stored procedures, and views, including SCD Type 1 and Type 2
• Developing with COPY INTO, PolyBase, and CETAS, and building parameterized Synapse/ADF pipelines (copy, data flows, triggers)
• Workload management, resource classes, statistics, materialized views, result‑set caching, and pool scaling/pausing
• ADLS Gen2 folder/file‑format design (Parquet/Delta), Entra ID access control, managed identities, and Key Vault
• Git and CI/CD in Azure DevOps or GitHub, including branching, PRs, and deploying Synapse artifacts and Databricks Asset Bundles
• Monitoring pipelines and workloads with Azure Monitor/Log Analytics and alerting, and troubleshooting failed jobs
• Strong written and verbal communication skill
• Using Microsoft lakehouse patterns, including interoperability and migration paths between Synapse and Databricks
• Databricks Certified Data Engineer Associate (Databricks Certified Data Engineer Professional a plus)
• Microsoft Certified: Azure Data Engineer Associate (DP-203) if already held
• Power BI connectivity to Synapse (Import vs. DirectQuery) and how model design affects dashboard performance