Senior Databricks Engineer - Lakehouse & AI Pipelines

AgileEngine, LLC.; Rosario; Presencial; ARS 60.365.000 - 128.275.000; Jornada completa

Ventajas ofrecidas por este puesto de trabajo

Flextime

Home office options

Mentorship program

Competitive compensation

Descripción de la vacante

AgileEngine is seeking a Senior Data Engineer in Rosario to design and operate batch and streaming data pipelines on Databricks using PySpark and Delta Lake. The role focuses on migrating legacy ETL workloads to a governed Lakehouse and implementing robust data governance with Unity Catalog.

Strong SQL and Python are essential for data modeling and analytics. You will work on medallion architectures (bronze/silver/gold) and collaborate with DevOps and analytics teams to ensure performance,

Formación

• 3+ years of professional data engineering experience with Apache Spark and cloud data architectures.
• Strong SQL and Python skills with data modeling across dimensional and Lakehouse patterns.
• Experience building data pipelines with Databricks, Spark (PySpark), and Delta Lake.
• Experience with streaming ingestion using Structured Streaming, Auto Loader, Kafka, or Event Hubs.
• Experience with workflow orchestration (Databricks Workflows, Airflow, or Azure Data Factory).
• Experience migrating legacy ETL/data warehouse workloads to Lakehouse while preserving data parity.
• Familiarity with Unity Catalog, data governance, access control, and PII handling.
• Experience with dbt or equivalent transformation framework.
• Familiarity with secure coding standards and security best practices.

Responsabilidades

• Design, build, and operate batch and streaming data pipelines on Databricks using PySpark, Delta Lake, and Databricks Workflows.
• Model and maintain a medallion architecture (bronze/silver/gold) for analytics, reporting, and ML.
• Migrate legacy ETL/data warehouse workloads onto the Lakehouse with minimal business disruption.
• Write clean, tested Python and SQL; enforce quality through code reviews and documentation.
• Optimize Spark jobs and Delta tables for performance and cost (partitioning, clustering, caching).
• Implement data quality, lineage, and governance using Unity Catalog and automated checks.
• Debug, troubleshoot, and resolve pipeline failures and production incidents.
• Collaborate with DevOps/analytics teams on observability, security, and compliance.

Conocimientos

Problem-solving

Collaboration

Communication

Herramientas

Databricks

Apache Spark (PySpark)

Delta Lake

Airflow

Azure Data Factory

Kafka

Event Hubs

Unity Catalog

dbt

Descripción del empleo

AgileEngine is seeking a Senior Data Engineer in Rosario to design and operate batch and streaming data pipelines on Databricks using PySpark and Delta Lake. The role focuses on migrating legacy ETL workloads to a governed Lakehouse and implementing robust data governance with Unity Catalog.

Strong SQL and Python are essential for data modeling and analytics. You will work on medallion architectures (bronze/silver/gold) and collaborate with DevOps and analytics teams to ensure performance,