Senior Data Engineer — Fintech Data Platform & AI

Allocate; Palo Alto (CA); Hybrid; USD 145,000 - 220,000; Full time

Benefits offered by this job

Medical, dental, and vision

401(k)

Discretionary PTO

Travel for offsites

Broadband internet

Code of Ethics

Job summary

Allocate is seeking a Senior Data Engineer to scale our data infrastructure and AI-driven features in a hybrid Palo Alto office. You will partner with the data lead to model core financial entities, ingest diverse sources, and build robust pipelines for analytics and product features.

You will work with a backend team (C#/.NET) and frontend (Node/Vue) to integrate data pipelines into our platform, delivering high-quality, scalable data services and AI capabilities in a startup environment.

Qualifications

• Experience designing and building large-scale data pipelines and storage solutions.
• Strong knowledge of AWS data services (S3, Redshift, Glue, Athena).
• Proficiency in SQL and data warehousing concepts; experience with lakehouses.
• Exposure to AI/ML data prep, feature stores, and vector embeddings is a plus.

Responsibilities

• Build and Extend Data Architecture: Extend data lakehouse on AWS and contribute to a knowledge graph and vector-based search.
• Develop Data Pipelines: Create robust ETL/ELT pipelines for batch and streaming data from multiple sources.
• Enable AI/ML Capabilities: Prepare data for ML, set up feature stores, and integrate model outputs into product features in real time.
• Engineering Excellence and Collaboration: Promote best practices, code quality, and cross-functional teamwork.
• Infrastructure and DevOps: Containerize data tools with Docker/Kubernetes on AWS EKS; implement CI/CD for data workflows.
• Continuous Improvement: Stay current with AWS offerings and open-source ML tools; evaluate new tech to improve reliability.

Skills

AWS Cloud

Data Modeling

SQL & Data Warehousing

Python / PySpark

ETL / ELT

Education

Bachelor's degree in Computer Science or equivalent

Tools

Docker

Kubernetes

Airflow

dbt

Snowflake

Databricks Delta Lake

Neo4j

Job description

Allocate is seeking a Senior Data Engineer to scale our data infrastructure and AI-driven features in a hybrid Palo Alto office. You will partner with the data lead to model core financial entities, ingest diverse sources, and build robust pipelines for analytics and product features.

You will work with a backend team (C#/.NET) and frontend (Node/Vue) to integrate data pipelines into our platform, delivering high-quality, scalable data services and AI capabilities in a startup environment.