Data engineer

Luza Tecnologia

United States

Remote

USD 110,000 - 160,000

Full time

3 days ago
Be an early applicant

A complete application in a minute

Tailored resume and cover letter, ready to send.

Get past ATS filters

Benefits offered by this job

Remote work
Food ticket
Health insurance
Training and certification vouchers
Work-life balance initiatives
Referral program
Birthday off
Give back day

Job summary

Luza Tecnologia ищет инженера данных с опытом Databricks для проектирования, построения и эксплуатации конвейеров данных в рамках архитектуры Lakehouse. В команде по данным вы будете тесно сотрудничать с аналитиками, дата-сайентистами и бизнес-коллегами.

Основной упор — платформа Databricks: ingestion, трансформации и надёжное обслуживание данных. Опыт Azure и Microsoft Fabric рассматривается как второстепенное требование и будет полезен для интеграции с широкой экосистемой данных.

Qualifications

  • 3+ лет опыта Data Engineer, 2+ года на Databricks в продакшене.
  • Сильные навыки PySpark, Spark SQL и Python.
  • Углубленные SQL, моделирование данных ( dimensional и Lakehouse ).
  • Опыт работы с Delta Lake и медальонной архитектурой.
  • Опыт Unity Catalog, управления доступом и governance.
  • Оркестрация конвейеров на Databricks (Jobs/Workflows, Delta Live Tables).
  • Git, CI/CD и TDD в data-проектах.
  • Знание Stream Processing (Structured Streaming, Auto Loader).
  • Azure Databricks и Azure Data Lake Storage Gen2; Azure Data Factory.
  • Безопасность: Microsoft Entra ID, Key Vault, managed identities, private networking.
  • Azure DevOps (Repos & Pipelines).

Responsibilities

  • Проектирование и реализация пакетных и потоковых конвейеров на Databricks.
  • Построение слоистой Lakehouse-архитектуры на Delta Lake.
  • Разработка декларативных пайплайнов и оркестрация workloads с Lakeflow.
  • Реализация governs с Unity Catalog: каталоги, разрешения, трассировка.
  • Моделирование данных для аналитического потребления BI и Data Science.
  • Документация решений и участие в стандартам инженерии команды.

Skills

Data Eng exp
PySpark
Spark SQL
Python
SQL & Data Modeling
Delta Lake
Unity Catalog
Databricks Orchestration
Git & CI/CD
Structured Streaming
Azure Databricks
Azure Data Factory
Azure Data Lake Storage Gen2
Azure DevOps
dbt
Terraform

Tools

dbt
Terraform
Azure DevOps

Job description

Описание:

We are looking for a Data Engineer with solid Databricks experience to design, build and operate data pipelines on a Lakehouse architecture. You will join a data team and work closely with analysts, data scientists and business teams. The focus is the Databricks platform: ingesting, transforming and serving reliable, governed and well-performing data. Experience with Azure and Microsoft Fabric is a secondary requirement, relevant for integration with the wider data ecosystem.

Задачи:

Design and implement batch and streaming data pipelines on Databricks, using PySpark and Spark SQL. Build and maintain a layered Lakehouse architecture (medallion: bronze, silver, gold) on Delta Lake. Develop declarative pipelines and orchestrate workloads with Lakeflow (Delta Live Tables, Jobs/Workflows). Implement data governance with Unity Catalog: catalogs, permissions, lineage and auditing. Model data for analytical consumption and make it available to BI and Data Science teams. Document solutions and contribute to the team's engineering standards and best practices.

Требования:

3+ years of experience as a Data Engineer, including at least 2 years on Databricks projects in production. Strong command of Apache Spark (PySpark and Spark SQL) and Python. Advanced SQL, including query optimization and data modeling (dimensional and Lakehouse). Hands-on experience with Delta Lake and the medallion architecture. Experience with Unity Catalog, access management and governance. Experience orchestrating pipelines on Databricks (Jobs/Workflows, Delta Live Tables). Working practice with Git, CI/CD and test-driven development in data projects. Knowledge of stream processing (Structured Streaming, Auto Loader). Azure Azure Databricks and Azure Data Lake Storage Gen2. Azure Data Factory for ingestion and orchestration. Security and identity: Microsoft Entra ID, Key Vault, managed identities and private networking. Azure DevOps (Repos and Pipelines). Будет плюсом: Databricks certifications (Data Engineer Associate or Professional); Experience with dbt, Terraform or other infrastructure-as-code tools; Experience migrating to Databricks from legacy platforms (SQL Server, Synapse, Hadoop); Knowledge of MLflow and of supporting Machine Learning and AI use cases

Условия:
  • Remote work
  • Food ticket
  • Health insurance
  • Training and access to certification vouchers
  • Work-life balance initiatives
  • Referral program
  • Birthday off
  • Give back day – enjoy a day to give back to society!
Get your free, confidential resume review.

or drag and drop your file here.

Similar jobs

Similar jobs worth comparing

Data engineer in financial services
Data engineer in financial services

Intellias • United States

Remote
USD 140,000 - 210,000
Data engineer in Lakehouse architectures
Data engineer in Lakehouse architectures

Teamative • United States

Remote
USD 90,000 - 140,000
data engineer for scalable data pipelines
data engineer for scalable data pipelines

EPAM • United States

On-site
USD 110,000 - 160,000
Data engineer
Data engineer

Cloudfide • United States

Remote
USD 120,000 - 180,000
E-learning budget
Ownership opportunities
Remote-first culture
Data engineer
Data engineer

emagine Group • United States

Remote
USD 46,000 - 66,000
Data engineer
Data engineer

AgileEngine, LLC. • United States

Hybrid
USD 120,000 - 160,000
Mentorship program
Tech Talks
Growth roadmap
+1
data engineer for a Lakehouse
data engineer for a Lakehouse

Data Smart Solutions • United States

On-site
USD 120,000 - 170,000
Performance-based bonus
Data engineer in operational risk management
Data engineer in operational risk management

Itransition • United States

Remote
USD 120,000 - 160,000
Corporate medical insurance
Gym membership
5 days of paid sick leave
Data Engineer (DATABRICKS / PYSPARK)
Data Engineer (DATABRICKS / PYSPARK)

Sigma Software • United States

Remote
USD 90,000 - 130,000
DBA in Databricks
DBA in Databricks

Nitka Technologies • United States

Remote
USD 100,000 - 170,000
Paid vacation
Holidays
Sick leaves
+1