Data Engineer

AspiringIT; Irvine (CA); On-site; USD 110,000 - 160,000; Full time

Job summary

AspiringIT in Irvine is seeking an experienced Data Engineer to design, build, and operate the pipelines powering our Prism platform on Databricks. You will move beyond bespoke scripts, enabling new sources via configuration files and scalable metadata-driven pipelines.

You will implement dbt transformations with testing and documentation, ensure governance with Unity Catalog, and build CI/CD workflows for high data quality and reliability.

Qualifications

• 3+ years of Data Engineering in production environments.
• Configuration-driven or metadata-driven data pipelines.
• Hands-on dbt development, testing, and docs.
• Strong SQL and Python skills.
• Experience with Databricks or Snowflake.
• Knowledge of orchestration tools (Airflow, Databricks Workflows, dbt Cloud).

Responsibilities

• Design and develop scalable, metadata-driven data pipelines.
• Build ingestion frameworks for batch, CDC, streaming, API and file sources.
• Develop dbt transformations with testing and documentation.
• Collaborate with data architects to translate requirements into data solutions.
• Implement governance and lineage using Unity Catalog.
• Build CI/CD pipelines and ensure data quality and observability.
• Troubleshoot and optimize end-to-end data workflows.

Skills

Data pipelines

dbt

SQL

Python

Databricks

Airflow

Cloud platforms

Tools

Unity Catalog

Terraform

Git

Kafka

dbt Cloud

Job description

About the Role:

We are looking for an experienced Data Engineer to design, build, and operate the pipelines that power our Databricks-based Prism platform. This role is for someone who has moved beyond one-off scripts and hand-built pipelines — you've built ingestion and transformation frameworks that are driven by configuration, not bespoke code, and you can stand up a new data source by writing a config file rather than a new pipeline from scratch.

Key Responsibilities

• Design and develop scalable, metadata-driven data pipelines.


• Build and maintain ingestion frameworks supporting batch, CDC, streaming, API, and file-based data sources.


• Develop transformation layers using dbt, including testing, documentation, and reusable components.


• Work closely with data architects and product owners to translate business requirements into data solutions.


• Implement governance and lineage using Unity Catalog.


• Build CI/CD pipelines and ensure high data quality, observability, and operational reliability.


• Troubleshoot and optimise end-to-end data workflows.

Required Skills & Experience

• 3+ years of Data Engineering experience in production environments.


• Strong experience with configuration-driven or metadata-driven data pipeline frameworks.


• Hands-on experience with dbt, including model development, testing, and documentation.


• Strong SQL and Python skills.


• Experience with cloud data platforms such as Databricks or Snowflake.


• Knowledge of orchestration tools such as Airflow, Databricks Workflows, or dbt Cloud.


• Ability to explain architecture decisions, troubleshooting approaches, and pipeline design best practices.

Preferred Qualifications

• Databricks, Delta Lake, and Unity Catalog experience.


• Terraform and Git-based CI/CD practices.


• Experience with Kafka, Structured Streaming, or Event Hubs.


• Background in Financial Services or Lending domains.


• Experience migrating legacy ETL solutions to cloud-based architectures.