Senior Databricks Engineer | Flexible Hours & Remote

Agileengine; Victoria; Híbrido; ARS 120.729.000 - 196.185.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Professional growth

Competitive USD-based compensation

A selection of exciting projects

Flextime

Descripción de la vacante

AgileEngine is seeking a Senior Data Engineer to design and operate batch and streaming data pipelines on Databricks using PySpark and Delta Lake. You will migrate legacy data workloads to a governed Lakehouse, modeling a medallion architecture that powers analytics and AI use cases.

You will also implement data quality with Unity Catalog, optimize Spark jobs, and collaborate across DevOps, platform, and analytics teams to ensure observability, security, and cost efficiency.

Formación

• 3+ years of professional data engineering experience.
• Hands-on with Databricks, Spark (PySpark), and Delta Lake in cloud architectures.
• Advanced SQL and Python with strong data modeling skills across dimensional and Lakehouse patterns.
• Experience with streaming ingestion and orchestration tools such as Structured Streaming, Auto Loader, Kafka, Databricks Workflows, Airflow, or Azure Data Factory.

Responsabilidades

• Design, build, and operate batch and streaming data pipelines on Databricks using PySpark, Delta Lake, and Databricks Workflows.
• Model and maintain a medallion (bronze/silver/gold) architecture serving analytics, reporting, and ML consumers.
• Migrate legacy ETL and data warehouse workloads onto the Lakehouse with validated data parity and minimal disruption.
• Use Claude or GitHub Copilot as a development accelerator, generating code scaffolding, writing/tests, documenting, and prototyping solutions.
• Write clean, well-tested Python and SQL; enforce quality via code reviews and documentation.
• Optimize Spark jobs and Delta tables for performance and cost (partitioning, clustering, caching, cluster sizing).
• Implement data quality, lineage, and governance with Unity Catalog and automated checks.
• Debug, troubleshoot, and resolve pipeline failures and production incidents.
• Collaborate with DevOps, platform, and analytics engineers on observability, security, and compliance.

Conocimientos

Databricks

Apache Spark (PySpark)

Delta Lake

SQL

Python

Data Modeling

Herramientas

Airflow

Terraform

Azure Data Factory

Descripción del empleo

AgileEngine is seeking a Senior Data Engineer to design and operate batch and streaming data pipelines on Databricks using PySpark and Delta Lake. You will migrate legacy data workloads to a governed Lakehouse, modeling a medallion architecture that powers analytics and AI use cases.

You will also implement data quality with Unity Catalog, optimize Spark jobs, and collaborate across DevOps, platform, and analytics teams to ensure observability, security, and cost efficiency.