Senior Data Engineer: Lakehouse, AI & End-to-End Pipelines

Brio Water Technology; Glendale Estates (AL); On-site; USD 120,000 - 180,000; Full time

Benefits offered by this job

Medical, Dental, Vision, LifeInsurance

401K Retirement Plan

Paid Vacation Time

Paid Holidays

and More!

Job summary

Brio Water Technology, a Home Organizers, Inc. company, seeks a hands-on data engineer to build and operate the full data platform—from ingestion and lakehouse governance to modeling, AI integration, and reporting.

You will own pipelines end‑to‑end and collaborate across business units to deliver trusted data products. The role emphasizes designing scalable data ingestion, managing Databricks lakehouse, implementing a governed semantic layer, and deploying ML/AI components with robust

Qualifications

• 8+ years in data engineering, including at least 3 owning a production cloud data platform end to end, from ingestion through the reporting layer.
• Hands‑on production experience with Databricks: Unity Catalog, Delta, Workflows/Lakeflow Jobs, and SQL warehouses.
• Expert dbt and dimensional modeling: layered projects, tests, documentation, and a semantic or metrics layer (MetricFlow, Metric Views, or LookML).
• Strong SQL and Python. Comfortable with Git, CI/CD, and infrastructure as code.
• Built governed BI on a semantic model (Omni, Sigma, Looker, or similar) with row‑level security, scheduled delivery, and alerting.
• Delivered both managed‑connector ingestion (Lakeflow Connect, Fivetran, Airbyte) and custom pipelines, including CDC and API sources.
• Shipped at least one pipeline that turns unstructured data (audio, documents, or images) into structured data using transcription, OCR, LLM classification, or vision models.
• Operated models in production with MLflow or an equivalent registry, including versioning, scoring jobs, and monitoring.
• Implemented data access controls: row filters, column masks, PII handling, and group‑based permissions synced from an identity provider.
• Solved entity resolution across multiple source systems (customers, locations, or employees).
• Built integrations that write back to production systems (in‑house applications, CMS, or SaaS APIs) with authorization checks, idempotency, rollback, and audit logging.
• Consumed APIs built by other teams and written clear requirements for what a data or AI integration needs from them.
• Able to work directly with senior business leaders to define metrics and explain trade‑offs in plain language.

Responsibilities

• Ingestion: Design, build, and operate batch and streaming pipelines from SaaS applications, databases, APIs, and custom in‑house systems, using managed connectors where they fit and custom code (including change data capture) where they do not.
• Lakehouse and governance: Build and administer a Databricks lakehouse (medallion architecture, Unity Catalog), including row filters, column masks, tagging, lineage, and identity‑provider group sync.
• Modeling and semantic layer: Develop dimensional models, entity resolution, and a governed metrics layer in dbt. Own testing, documentation, and data quality.
• Unstructured data and ML: Build pipelines that turn audio, video, documents, images, and sensor or edge‑device telemetry into structured data using transcription, OCR, LLM classification, and computer vision. Deploy, version, and monitor models with MLflow; build and maintain vector search indexes.
• AI integration: Build and maintain LLM integrations (MCP servers, retrieval, text‑to‑SQL) with both read and write capabilities, enforcing user‑level permissions, approval workflows, and audit logging.
• APIs: Define integration requirements for APIs built by our application engineering team, and build against them.
• Reporting: Build governed dashboards, reports, scheduled delivery, and alerting on a semantic‑model BI platform.
• Observability: Monitor pipeline freshness, data quality, model performance, and index health, with alerting that surfaces problems early.

Skills

Databricks

dbt

SQL

Python

Git

CI/CD

infrastructure as code

MLflow

data modeling

entity resolution

data governance

Tools

Unity Catalog

Delta

Lakeflow Jobs

Fivetran

Airbyte

Looker

MLflow

Job description

Brio Water Technology, a Home Organizers, Inc. company, seeks a hands-on data engineer to build and operate the full data platform—from ingestion and lakehouse governance to modeling, AI integration, and reporting.

You will own pipelines end‑to‑end and collaborate across business units to deliver trusted data products. The role emphasizes designing scalable data ingestion, managing Databricks lakehouse, implementing a governed semantic layer, and deploying ML/AI components with robust