Senior Data Platform Engineer - Remote & Scalable Pipelines

Parafin, Inc.; San Francisco (CA); On-site; USD 220,000 - 265,000; Full time

Benefits offered by this job

Work from home flexibility

Unlimited PTO

Free lunches

401(k)

Equity grant

Paid parental leave

Employee assistance program

Job summary

Parafin, Inc. is seeking a Senior Software Engineer to lead the development of our Data Platform within the Infrastructure team.

You will design, build, and maintain data pipelines and lakehouse infrastructure, collaborating with Data Science, Platform, and Product Engineering to enable data-driven decisions. You will own Databricks workspaces, Terraform configurations, and end-to-end data ingestion, transformation, and serving.

Qualifications

• 4+ years of software engineering with a focus on data infrastructure and distributed systems.
• Advanced proficiency in Python and SQL.
• Expertise with cloud data stacks—AWS, Databricks, Airflow, and lakehouse architectures.
• Hands-on experience with Hadoop, Hive, Kafka, Delta Lake, and distributed query engines (Trino/Presto).
• Familiarity with data versioning, lineage, partitioning, and clustering.

Responsibilities

• Design and build scalable data pipelines and lakehouse infrastructure.
• Improve data platform development experience with self-service tooling and documentation.
• Own core data pipelines and models powering dashboards, ML models, and products.
• Manage Databricks workspaces and Terraform-based infrastructure for reliability and cost control.
• Lead data quality, testing, observability, and migration efforts (e.g., Databricks to dbt).
• Partner with Data Science to support data-driven product development and decisions.
• Collaborate with backend engineers and data scientists to design performant data models.

Skills

Python

SQL

Data pipelines

Distributed systems

Data infrastructure

Mentorship

Tools

Databricks

Airflow

Terraform

Delta Lake

Hadoop

Hive

Kafka

dbt

Trino/Presto

Job description

Parafin, Inc. is seeking a Senior Software Engineer to lead the development of our Data Platform within the Infrastructure team.

You will design, build, and maintain data pipelines and lakehouse infrastructure, collaborating with Data Science, Platform, and Product Engineering to enable data-driven decisions. You will own Databricks workspaces, Terraform configurations, and end-to-end data ingestion, transformation, and serving.