Data Engineer - Databricks Migration

Randstad Enterprise; United States; On-site; USD 171,924,000 - 183,386,000; Full time

Job summary

Randstad Enterprise is seeking a mid-level Databricks data engineer to ingest and transform data for enterprise BI solutions. The role focuses on building silver and gold tables in Databricks, leveraging Health Catalyst on Microsoft SQL Server, and partnering with BI analysts to deliver reliable reporting.

Requires 4+ years in SQL, ETL with Azure Databricks, DBSQL, T-SQL, SSIS, and experience with various data sources.

Qualifications

• 4+ years of related experience.
• Strong SQL and T-SQL skills.
• Experience with Azure Databricks and DBSQL.
• Experience with data warehousing concepts.

Responsibilities

• Ingests: Design and implement Databricks ingestion processes using customer tools and sources.
• Design and Build: Create silver and gold tables in Databricks based on Health Catalyst on SQL Server.
• Collaborate with BI analysts and developers to design and build tables.
• Design and Code Reviews: Prepare for and participate in design and code reviews.
• Test and Troubleshoot: Participate in testing, ensure reporting functionality and fix issues.
• Compliance: Ensure adherence to HIPAA and corporate privacy policies.

Skills

SQL

ETL

Data Ingestion

Database tuning

Data warehousing

Power BI

HIPAA compliance

Tools

Health Catalyst

Epic

Azure Databricks

Job description

*This is a mid level \"do-er\" role. Not senior / architect level*

Start: 2 weeks from date of offer

Length of contract: 6 months

Hourly Pay: 60-64 p/hr on W2

*W2 Contract Only*

Responsibilities:

• Ingests: Design and implement Databricks ingestion processes, leveraging Databricks skills, customer tools and skills with different types of sources.
• Design and Build: Design silver and gold tables in Databricks based on existing solutions deployed in Health Catalyst installed on Microsoft SQL Server.
• Collaborate and Develop: Work collaboratively with Business Intelligence analysts and Business Intelligence developers to design and build silver and gold tables.
• Design and Code Reviews: Prepare for and participate in design and code reviews according to customer design standards.
• Test and Troubleshoot: Participate in testing methodologies, ensuring the functionality and integrity of our reporting solutions. Troubleshoot and resolve errors and solve complex technical problems.
• Compliance: Ensure adherence to corporate policies and privacy regulations like HIPAA.

Experience/skills:

• SQL Proficiency: Strong SQL experience writing queries, creating tables, and performing updates with Transact SQL and DB SQL.
• ETL Expertise: Strong experience writing ETL with Azure Databricks, DBSQL, Microsoft SQL Server, Transact SQL, SSIS.
• Data Ingestion: Experience ingesting data from many different types of sources (RDBMS, XML files, JSON files, Flat files, APIs)
• Database Expertise: Experience with database management systems, including performance optimizing and tuning and security
• Design Experience: Experience with data warehousing methodologies and data model development
• Additional Experience: Experience with Power BI, Health Catalyst, and Epic is a plus
• Communication/Collaboration: Speak up quickly when something is discovered, listen and work with others to solve problems, share lessons learned from prior migrations.

Required criteria:

• Experience: Minimum of 4 years of related experience