Remote .Net Data Analytics Engineer

Toptal; Ciudad de Mendoza; Remote; ARS 1,500,000 - 2,100,000; Full time

Job summary

Toptal is seeking a data engineer to design and optimize scalable data pipelines. You will implement Medallion architecture in Python, ensure cross-cloud compatibility with Parquet/Iceberg/Delta Lake, and embed analytics with DuckDB in ETL workflows.

You will also optimize PostgreSQL schemas, leverage SQLite for lightweight storage, and scale workloads using Databricks or Spark in distributed processing environments.

Qualifications

• Advanced SQL with PostgreSQL expertise is required.
• Experience building analytical pipelines using Python and DuckDB.
• Proficiency with open storage formats (Parquet, Iceberg, Delta Lake).
• Proven track record designing data lakes with Medallion architecture.
• Familiarity with Databricks/PySpark for distributed processing.

Responsibilities

• Pipeline Design: Build resilient data processing pipelines in Python following Medallion architecture (Bronze/Silver/Gold).
• Open Storage Management: Standardize data storage with Parquet, Iceberg, and Delta Lake for cross-cloud compatibility.
• Embedded Analytics: Use DuckDB integrated with Python for fast localized data processing within ETL workflows.
• Database Operations: Architect and optimize production PostgreSQL schemas; use SQLite for embedded storage when needed.
• Distributed Compute: Scale ETL and analytics with Databricks or Apache Spark.

Skills

SQL proficiency

Python programming

Analytical thinking

Tools

PostgreSQL

SQLite

Parquet

Apache Iceberg

Delta Lake

Medallion architecture

Databricks

PySpark

DuckDB

Job description

The Role:

Builds performant data processing pipelines, manages relational databases, and constructs cloud-portable data lake environments using modern open storage formats and embedded analytical engines.

Responsibilities:

Pipeline Design: Construct resilient data processing pipelines in Python using Medallion architecture principles (Bronze/Silver/Gold) to transform raw data into analytics-ready assets.

Open Storage Management: Standardize data storage using open formats including Parquet, Apache Iceberg, and Delta Lake to ensure cross-cloud compatibility.

Embedded Analytics: Leverage DuckDB integrated with Python for fast, lightweight, and performant localized data processing within ETL workflows.

Database Operations: Architect and optimize production PostgreSQL schemas while utilizing SQLite for embedded or lightweight localized storage needs.

Distributed Compute: Scale heavy ETL, data transformation, and analytics workloads using Databricks or Apache Spark

Requirements:

Advanced SQL proficiency with deep PostgreSQL expertise and working knowledge of SQLite.

Hands-on experience building robust analytical pipelines using Python and DuckDB.

Mastery of open table and storage formats (Parquet, Apache Iceberg, Delta Lake).

Proven track record designing data lakes using the Medallion data processing architecture.

Familiarity with the Databricks / PySpark ecosystem for distributed processing.