Sr Data Engineer (Data Bricks)

BinaryBees Business Solutions LLC; Chicago (IL); Ibrido; USD 120.000 - 160.000; Tempo pieno

Descrizione del lavoro

BinaryBees Business Solutions LLC in Chicago, IL is seeking a Senior Data Engineer to join a hybrid team building a brand-new Azure Databricks platform. You will design and deliver end-to-end pipelines, governance, and models on a greenfield data platform in a direct-hire, full-time role.

This hands-on role requires expert Python, SQL, Spark, and Delta Lake skills, with ownership of pipelines from ingestion through analytics outputs, plus mentoring junior engineers and collaborating with BI and

Competenze

• Bachelor's degree in a related field.
• Experience building end-to-end data pipelines on modern platforms.
• Hands-on Databricks experience including Delta Lake and Unity Catalog.

Mansioni

• Lead design and build of scalable data pipelines and models on Azure Databricks
• Own pipelines end-to-end from ingestion to transformation and modeling
• Migrate legacy data warehouses to a centralized Databricks lakehouse
• Write code daily in Python, SQL, and Spark
• Establish CI/CD standards and automated data testing
• Help shape data modeling and governance standards
• Troubleshoot and optimize platform performance
• Mentor junior engineers and collaborate with cross-functional teams

Conoscenze

Azure Databricks

Python

SQL

Spark

ETL/ELT pipelines

Delta Lake

Data modeling

CI/CD

Git workflows

Problem solving

Formazione

Bachelor's degree in Computer Science, Engineering, Data Science, or related field

Strumenti

Databricks Workflows

GitHub

Azure DevOps

Terraform

Descrizione del lavoro

Primary Location: Chicago, IL (Hybrid – 3 days onsite / 2 days remote)

Position Type: Full-Time, Direct Hire

Must Be Citizen or Green Card

OVERVIEW

A Senior Data Engineer. This is a full-time, direct hire role in Chicago, IL, working a hybrid schedule with a minimum of three days onsite per week. This role exists as part of a multi-year initiative to transform our client's Data & AI function, building a modern data platform that enables speed-to-insight and makes the organization truly data-driven.

You'll be one of the core builders of a brand-new Azure Databricks platform, designing and delivering the pipelines, transformations, and models that everything else runs on. The environment is greenfield: Databricks is stood up, but no production pipelines exist yet, so you'll help set the standards from day one. This is a hands-on senior individual contributor role, not a people management, architecture, AI/LLM, or BI/reporting role. The architecture is defined; your job is to build it well and raise the technical bar of a growing team (expanding to 25–30) that is largely BI-focused today.

The work carries strong executive sponsorship and enterprise-wide visibility, with the chance to influence engineering practices across the organization. The interview process includes a live technical assessment covering Python, SQL, and practical data pipeline logic.

WHAT YOU BRING TO THE ROLE. (IDEAL EXPERIENCE)

• Bachelor's degree in Computer Science, Engineering, Data Science, or a related field
• 6+ years of hands‑on data engineering experience, with a track record of owning pipelines end to end: ingestion, validation, transformation, and governance
• Real, hands‑on Databricks experience (Azure preferred; AWS or GCP acceptable), including Delta Lake, Unity Catalog, Databricks SQL, Spark clusters, and Workflows/Jobs
• Strong coding skills in Python, SQL, and Apache Spark that go well beyond basic scripting
• Proven experience designing and building scalable ETL/ELT pipelines, including reusable, metadata-driven pipelines
• Experience with object storage, lakehouse engines, orchestration tools, and streaming or CDC pipelines
• Solid data modeling, schema design, and performance tuning experience
• Working knowledge of Git-based workflows, CI/CD, test automation, and monitoring and alerting
• Strong troubleshooting skills and a solid grasp of engineering fundamentals: scalability, reliability, and maintainability
• The ability to clearly explain what you built, why it mattered, and how it solved a business or platform problem

Preferred Experience building or migrating a data platform from the ground up

• Infrastructure as Code (Terraform, Bicep, ARM) and Governance as Code experience
• Multi‑cloud background with depth in Azure
• Legacy data warehousing and/or Hadoop experience
• Deep Databricks exposure beyond basic pipelines, such as enterprise‑scale work and optimization
• A track record of mentoring and upskilling peers

WHAT YOU'LL DO. (SKILLS USED IN THIS POSITION)

• Lead the design and build of scalable data pipelines and models in Azure Databricks on a brand-new platform with no production code yet
• Own pipelines end to end, from ingesting source systems to transforming and modeling data to producing outputs for downstream analytics, BI, and AI/ML use cases
• Refactor and migrate multiple legacy data warehouses into the centralized Databricks lakehouse
• Work at the code level daily in Python, SQL, and Spark, building pipelines with Databricks Asset Bundles, Spark workflows, and API‑driven jobs
• Establish engineering standards for CI/CD (GitHub/Azure DevOps), automated data testing, monitoring, and alerting
• Help shape and enforce standards for data modeling and governance
• Troubleshoot and optimize performance across the platform
• Mentor and coach junior engineers, conduct code reviews, and lead by example in design, coding, and testing practices
• Partner with Data Engineering, BI, AI, Finance, and product owners, and help translate technical work into business impact