Data Pipelines Architect (Python, Spark, DuckDB)

Toptal; Ciudad de Mendoza; Remote; ARS 900,000 - 1,300,000; Full time

Job summary

Toptal is seeking a data engineering professional to design resilient ETL pipelines and standardized data lakes. You will implement Medallion architecture, leverage Parquet/Iceberg/Delta formats, and perform embedded analytics with Python and DuckDB.

You will optimize PostgreSQL schemas and support lightweight SQLite storage, while enabling distributed processing with Databricks or Spark in a cross-cloud setup.

Qualifications

• Advanced SQL with PostgreSQL expertise and SQLite knowledge.
• Hands-on experience building analytical pipelines with Python and DuckDB.
• Mastery of open formats like Parquet, Iceberg, and Delta Lake.
• Proven track record designing data lakes using Medallion architecture.
• Familiarity with Databricks and PySpark for distributed processing.

Responsibilities

• Pipeline design: build robust data processing pipelines in Python using Medallion architecture.
• Open storage management: standardize data in Parquet, Iceberg, Delta Lake for cross-cloud use.
• Embedded analytics: use DuckDB with Python for fast, localized ETL analytics.
• Database operations: architect and optimize PostgreSQL schemas and lightweight SQLite storage.
• Distributed compute: scale ETL and analytics with Databricks or Spark.

Skills

Advanced SQL (PostgreSQL)

Python + DuckDB

Open table/storage formats

Medallion architecture

Databricks / PySpark

Job description

Toptal is seeking a data engineering professional to design resilient ETL pipelines and standardized data lakes. You will implement Medallion architecture, leverage Parquet/Iceberg/Delta formats, and perform embedded analytics with Python and DuckDB.

You will optimize PostgreSQL schemas and support lightweight SQLite storage, while enabling distributed processing with Databricks or Spark in a cross-cloud setup.