A complete application in a minute
Tailored resume and cover letter, ready to send.
Luza Tecnologia ищет инженера данных с опытом Databricks для проектирования, построения и эксплуатации конвейеров данных в рамках архитектуры Lakehouse. В команде по данным вы будете тесно сотрудничать с аналитиками, дата-сайентистами и бизнес-коллегами.
Основной упор — платформа Databricks: ingestion, трансформации и надёжное обслуживание данных. Опыт Azure и Microsoft Fabric рассматривается как второстепенное требование и будет полезен для интеграции с широкой экосистемой данных.
We are looking for a Data Engineer with solid Databricks experience to design, build and operate data pipelines on a Lakehouse architecture. You will join a data team and work closely with analysts, data scientists and business teams. The focus is the Databricks platform: ingesting, transforming and serving reliable, governed and well-performing data. Experience with Azure and Microsoft Fabric is a secondary requirement, relevant for integration with the wider data ecosystem.
Design and implement batch and streaming data pipelines on Databricks, using PySpark and Spark SQL. Build and maintain a layered Lakehouse architecture (medallion: bronze, silver, gold) on Delta Lake. Develop declarative pipelines and orchestrate workloads with Lakeflow (Delta Live Tables, Jobs/Workflows). Implement data governance with Unity Catalog: catalogs, permissions, lineage and auditing. Model data for analytical consumption and make it available to BI and Data Science teams. Document solutions and contribute to the team's engineering standards and best practices.
3+ years of experience as a Data Engineer, including at least 2 years on Databricks projects in production. Strong command of Apache Spark (PySpark and Spark SQL) and Python. Advanced SQL, including query optimization and data modeling (dimensional and Lakehouse). Hands-on experience with Delta Lake and the medallion architecture. Experience with Unity Catalog, access management and governance. Experience orchestrating pipelines on Databricks (Jobs/Workflows, Delta Live Tables). Working practice with Git, CI/CD and test-driven development in data projects. Knowledge of stream processing (Structured Streaming, Auto Loader). Azure Azure Databricks and Azure Data Lake Storage Gen2. Azure Data Factory for ingestion and orchestration. Security and identity: Microsoft Entra ID, Key Vault, managed identities and private networking. Azure DevOps (Repos and Pipelines). Будет плюсом: Databricks certifications (Data Engineer Associate or Professional); Experience with dbt, Terraform or other infrastructure-as-code tools; Experience migrating to Databricks from legacy platforms (SQL Server, Synapse, Hadoop); Knowledge of MLflow and of supporting Machine Learning and AI use cases