Lead Data Engineer - AWS Databricks Lakehouse (Remote)

Virisha LLC

Chicago (IL)

Hybrid

USD 150,000 - 180,000

Full time

6 days ago

Be an early applicant

Application generator

Don’t send a generic resume — generate a resume and cover letter tailored to this exact role.

Get past ATS filters

Job summary

Virisha LLC is seeking a Lead Data Engineer to design and implement an AWS Databricks Lakehouse platform, shaping analytics strategy and data governance. You will partner with IT, analytics, and business teams to deliver scalable solutions and drive decisions.

The role requires 10 years in IT with 4–6 years in data pipelines, Databricks on AWS, and strong SQL/Python skills. A leadership mindset and experience mentoring a team are a plus for this contract-to-hire position.

Qualifications

10 years of IT experience with deep data engineering & ETL.
4-6 years hands-on experience designing, building, and scaling data pipelines.
Strong hands-on experience with Databricks (Spark, Delta Lake, Unity Catalog) and AWS (S3, IAM, networking).
Proficiency in Databricks architecture combining data lakes and data warehouses.
Familiar with Unity Catalog governance, DLT pipelines, Databricks SQL, and workflow orchestration.
Expertise in SQL, Python, data modeling, and distributed processing.
Experience with CI/CD, DevOps, and modern data architecture (Lakehouse, medallion).
Agile analytics collaboration across stakeholders and strong communication.

Responsibilities

Design and lead enterprise-grade pipelines for batch and streaming data processing.
Own the full ETL/ELT lifecycle from design to production support.
Drive data engineering standards, best practices, and architecture across the data ecosystem.
Translate business requirements into scalable, high-performance data solutions.
Mentor a small team while remaining hands-on.
Ensure data reliability, governance, and cost optimization.