Databricks Data Engineer - Remote Lakehouse & Streaming

AgileEngine; San Carlos de Bariloche; Presencial; ARS 2.000.000 - 3.000.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Growth without limits

Competitive compensation

Flexibility: remote work with flexible

Meaningful, modern projects

Collaborative culture

Well-being & support

Descripción de la vacante

AgileEngine is seeking a Middle Data Engineer to modernize a 15-year-old data warehouse into a governed Databricks Lakehouse. You will build batch and streaming pipelines with PySpark and Delta Lake, following a medallion architecture across bronze, silver, and gold layers.

You will also leverage AI tools like Claude and GitHub Copilot to accelerate development, write tests, create docs, and prototype solutions while ensuring data quality and governance across pipelines.

Formación

• 3+ years of professional data engineering experience with Spark and cloud data architectures.
• Strong hands-on with Databricks, PySpark, and Delta Lake.
• Advanced SQL and Python, with data modeling skills for lakehouse patterns.
• Experience with Structured Streaming, Auto Loader, Kafka, or Event Hubs.
• Experience with orchestration (Databricks Workflows, Airflow, or Azure Data Factory).
• Experience migrating legacy platforms and ensuring data hygiene.
• Strong problem-solving, collaboration, and communication skills.
• Familiarity with Unity Catalog, data governance, and PII handling.
• Experience with dbt or equivalent transformation framework.
• Familiarity with secure coding standards and security best practices.
• Experience delivering production data platforms at scale.

Responsabilidades

• Design, build, and operate batch and streaming pipelines on Databricks using PySpark and Delta Lake.
• Model and maintain medallion architecture (bronze/silver/gold) for analytics and ML users.
• Migrate legacy ETL workloads to the Lakehouse with minimal disruption.
• Use Claude or Copilot to speed up development, tests, docs, and prototyping.
• Write clean Python and SQL; enforce high standards via reviews and docs.
• Optimize Spark jobs and Delta tables for performance and cost.
• Implement data quality, lineage, and governance with Unity Catalog and validations.
• Debug, troubleshoot, and resolve production pipeline incidents.
• Collaborate with DevOps, platform, and analytics engineers on observability and security.

Conocimientos

Apache Spark

PySpark

Databricks

Delta Lake

SQL

Python

Structured Streaming

Auto Loader

Kafka

Event Hubs

Databricks Workflows

Airflow

Azure Data Factory

dbt

Unity Catalog

Data governance

PII handling

Security best practices

Production data platforms

English (Upper-intermediate)

Herramientas

Terraform

PostgreSQL

Descripción del empleo

AgileEngine is seeking a Middle Data Engineer to modernize a 15-year-old data warehouse into a governed Databricks Lakehouse. You will build batch and streaming pipelines with PySpark and Delta Lake, following a medallion architecture across bronze, silver, and gold layers.

You will also leverage AI tools like Claude and GitHub Copilot to accelerate development, write tests, create docs, and prototype solutions while ensuring data quality and governance across pipelines.