Tennessee Oncology is seeking a hands-on Data Engineer to build and support reliable data solutions that power reporting, analytics, and business intelligence. This onsite role in Nashville, TN focuses on delivering governed, high-quality datasets through a lakehouse approach in Databricks, along with strong integration and transformation work that supports Power BI Report Server reporting needs.
What You’ll Do
- Design, build, test, deploy, and support data ingestion and transformation pipelines in Databricks using SQL, Python, PySpark, notebooks, and workflows.
- Connect and integrate data from relational databases, APIs, files, cloud platforms, vendor systems, and other enterprise sources.
- Profile, clean, standardize, enrich, and validate data to ensure downstream reporting and analytical datasets are accurate, consistent, and usable.
- Develop and maintain scalable lakehouse structures using Parquet and Delta Lake, including partitioning, schema management, table design, and file optimization.
- Create and maintain SQL queries, views, stored procedures, functions, and transformation data models used for reconciliation, reporting, and analysis.
- Build and maintain curated datasets, semantic models, reports, dashboards, and visualizations for Power BI Report Server, partnering with report developers and business users on presentation accuracy, metric consistency, reliable refreshes, and performance.
- Implement data quality controls such as reconciliation checks, exception handling, logging, and alerting across pipelines.
- Investigate pipeline failures, unexpected results, refresh issues, and source-system changes, coordinating corrective action to completion.
- Monitor scheduled jobs, data freshness, storage utilization, and query performance, improving operational reliability, scalability, and cost efficiency.
- Apply established standards for architecture, naming, metadata, lineage, retention, access, privacy, and lifecycle management.
- Protect sensitive and regulated information using least-privilege access, secure data handling, appropriate masking, and compliance requirements.
- Collaborate with business users and technical teams to define source behavior, business rules, reporting needs, and acceptance criteria.
- Document source-to-target mappings, business rules, transformations, dependencies, operational procedures, and troubleshooting guidance.
- Use version control, code review, testing, deployment, and change-management practices to move data solutions safely between environments.
- Identify opportunities to standardize, automate, and simplify recurring data work to reduce manual preparation and reporting effort.
- Support production data operations and participate in after-hours troubleshooting or scheduled implementation activities when needed.
What You Bring
- Strong working knowledge of Databricks lakehouse concepts, including notebooks, clusters or serverless compute, jobs/workflows, SQL Warehouses, and Delta Lake.
- Strong SQL development skills: complex joins, common table expressions, window functions, query optimization, data reconciliation, and relational data modeling.
- Experience with Python and/or PySpark for ingestion, transformation, validation, and automation.
- Knowledge of Parquet structure and behavior, including columnar storage, compression, schema evolution, partitioning, and small-file performance considerations.
- Understanding of dimensional modeling, fact and dimension tables, star schemas, slowly changing dimensions, and semantic model design.
- Working knowledge of Power BI Report Server, Power BI Desktop, Power Query, DAX, dataset refresh, gateways/connectivity, security, and report performance.
- Understanding of ETL and ELT, including batch processing, incremental loading, change data capture, orchestration, dependency management, and restartable pipeline design.
- Knowledge of data quality, governance, metadata, lineage, master/reference data, retention, privacy, and role-based access principles.
- Ability to trace data end-to-end from source through transformation to final reporting and explain discrepancies clearly.
- Troubleshooting, analytical, documentation, organization, and time-management skills.
- Ability to manage competing priorities, communicate risks or blockers, and drive assigned work through completion.
- Ability to explain technical data concepts and findings clearly to technical and non-technical audiences.
- Self-motivated with judgment to ask questions, validate assumptions, and elevate concerns when needed.
- Education: Associate’s or Bachelor’s degree in Information Technology, Computer Science, Data Engineering, Analytics, or a related field, or at least 4 years of equivalent experience in lieu of degree.
- Experience: At least 3 years of experience in data engineering, business intelligence, data integration, database development, or a comparable role.
- Demonstrated hands-on experience developing SQL-based data solutions and supporting production data pipelines.
Technologies You’ll Work With
- Databricks, SQL, Python, PySpark, notebooks, workflows
- Parquet, Delta Lake, Power BI Report Server, Power BI Desktop, Power Query, DAX, SQL Warehouses
- Microsoft SQL Server, ETL, ELT
- Power BI, Amazon APIs, cloud platforms, APIs, Microsoft Fabric, Azure Data Engineer
Location and Work Conditions
- Location: Nashville, TN (onsite)
- Physical requirements: work primarily performed in an office or remote-office environment requiring extended use of a computer.
- Must be willing and able to travel to clinic or business locations when necessary.