Senior Databricks Data Engineer / Lakehouse Architect

Bilink Corp; Chicago (IL); Hybrid; USD 110,000 - 160,000; Full time

Benefits offered by this job

401(k) with match

Health and dental insurance

Paid time off

Job summary

Bilink Corp in Chicago seeks a Senior Databricks Data Engineer / Lakehouse Architect to design, build, and run a production Databricks lakehouse on Azure, using SAP as a data source.

This hybrid role collaborates with analytics and IT, focusing on governance, security (Unity Catalog), and performance tuning.

You will own architecture end to end, mentor engineers, and ensure data quality and scalable platform capabilities.

Qualifications

• 7+ years of data engineering experience
• 4+ years hands-on experience with Databricks on production projects
• Azure strongly preferred; AWS or GCP considered
• Strong Python and SQL skills with data modeling and warehousing experience
• Clear communication with technical and non-technical stakeholders
• Must reside in Chicago area and be able to work from River North office 3 days a week
• Authorized to work in the US; visa sponsorship considered for exceptional candidates
• No corp-to-corp; direct W2 employment with Bilink

Responsibilities

• Build ETL/ELT pipelines on Databricks using Spark and PySpark
• Implement Delta Lake and medallion architecture
• Support incremental loads and CDC using MERGE
• Set up governance and security with Unity Catalog
• Tune for performance and cost while maintaining data quality

Skills

Databricks

Spark

PySpark

Delta Lake

Unity Catalog

MERGE

Python

SQL

Data modeling

Data warehousing

Communication

Education

Bachelor’s or Master’s in Computer Science or Engineering or related field

Tools

Azure

Databricks

Spark

PySpark

Delta Lake

Unity Catalog

Job description

Bilink Corp is seeking a Senior Databricks Data Engineer / Lakehouse Architect to build and operate a Databricks lakehouse in production. In this hybrid Chicago role, you will help establish a Databricks platform on Azure, with SAP serving as a primary data source during the initial engagement. The work spans ETL/ELT delivery, governance, security, and ongoing performance and cost tuning.

Role overview

• Own the work from architecture through delivery: design, build, and run the lakehouse in production
• Serve as a trusted technical voice with client business, analytics, and IT teams
• Set engineering standards and mentor other engineers

Responsibilities

• Build ETL/ELT pipelines on Databricks using Spark and PySpark
• Implement Delta Lake and a medallion architecture approach
• Support incremental loads and CDC using MERGE
• Set up governance and security with Unity Catalog
• Tune for performance and cost while keeping data quality high

Requirements

• 7+ years in data engineering, including 4+ years hands-on with Databricks on production projects
• Experience designing a data platform, not only building pipelines inside a single system
• Azure strongly preferred (AWS or GCP considered)
• Strong Python and SQL, with solid data modeling and data warehousing experience
• Clear communication with technical and non-technical stakeholders
• Bachelor’s or Master’s in Computer Science, Engineering, or a related field, or equivalent experience
• Live in the Chicago area and be able to work from the River North office 3 days a week
• Authorized to work in the US (visa sponsorship considered for an exceptional candidate already based in the Chicago area)
• No C2C (corp-to-corp) or third-party vendors; direct W2 employee of Bilink

Technologies

• Databricks, Spark, PySpark, Delta Lake, Unity Catalog
• MERGE, Python, SQL
• Azure (plus AWS and GCP experience considered)
• Medallion architecture, CDC

Benefits

• 401(k) with match
• Health and dental insurance
• Paid time off

Nice to have

• SAP data integration experience (S/4HANA, BW, HANA) into a lakehouse
• Consulting experience
• Databricks certification (Data Engineer Professional or Associate)

Compensation and location

• Location: Chicago, IL (hybrid), River North office
• Salary: USD 110,000 - 160,000 per year

Hiring process

• Short written screening (application questions)
• 20-minute intro call
• On-site at the River North office: a 40-minute case study, then a presentation and technical discussion with the team