Senior Databricks Engineer – Remote-Optional & Growth

AgileEngine, LLC.; San Carlos de Bariloche; Híbrido; ARS 211.276.000 - 301.823.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Professional growth

Competitive USD-based compensation

Flextime

Descripción de la vacante

AgileEngine is seeking a Senior Data Engineer to build batch and streaming data pipelines on Databricks using PySpark and Delta Lake. You will migrate legacy data warehouse workloads to a governed Lakehouse, modeling a medallion architecture to power analytics and AI use cases.

Strong SQL, Python, and experience with Unity Catalog governance round out the role. You will design, implement, and optimize data pipelines while collaborating with DevOps, platform, and analytics engineers.

Formación

• 3+ years of professional experience in data engineering.
• Strong expertise with Apache Spark and cloud-based data architectures.
• Proficient in SQL and Python for data modeling and transformation.
• Experience with streaming ingestion using Structured Streaming, Auto Loader, Kafka, or Event Hubs.
• Experience with workflow orchestration (Databricks Workflows, Airflow, or Azure Data Factory).
• Experience with legacy platform migrations, ETL modernization, or data hygiene when porting old systems.

Responsabilidades

• Design, build, and operate batch and streaming data pipelines on Databricks using PySpark, Delta Lake, and Databricks Workflows.
• Model and maintain a medallion (bronze/silver/gold) architecture serving analytics and ML consumers.
• Migrate legacy ETL and data warehouse workloads onto the Lakehouse with validated data parity and minimal disruption.
• Develop with code scaffolding and tests using Claude or GitHub Copilot, and produce documentation.
• Write clean Python and SQL with high-quality code reviews and documentation.
• Optimize Spark jobs and Delta tables for performance and cost, including partitioning and caching.
• Implement data quality, lineage, and governance controls with Unity Catalog.

Conocimientos

Databricks

PySpark

SQL

Python

Unity Catalog

Medallion architecture

Data modeling

Debugging

Herramientas

Databricks

Apache Spark

Delta Lake

Kafka

Airflow

Azure Data Factory

dbt

Descripción del empleo

AgileEngine is seeking a Senior Data Engineer to build batch and streaming data pipelines on Databricks using PySpark and Delta Lake. You will migrate legacy data warehouse workloads to a governed Lakehouse, modeling a medallion architecture to power analytics and AI use cases.

Strong SQL, Python, and experience with Unity Catalog governance round out the role. You will design, implement, and optimize data pipelines while collaborating with DevOps, platform, and analytics engineers.