Data Engineer: Build scalable pipelines & cloud data lakes

Air Apps; San Francisco (CA); In loco; USD 140.000 - 190.000; Tempo pieno

Vantaggi offerti da questo lavoro

Apple hardware ecosystem for work.

Annual Bonus

Medical Insurance (vision & dental)

Disability insurance - short and long‑

401k up to 4%

Air Conference

Transportation budget

Free meals at the hub

Gym membership

Descrizione del lavoro

Air Apps is seeking a Data Engineer to design, build, and optimize data pipelines, warehouses, and lakes. You’ll work with analysts, scientists, and engineers to create scalable data infrastructure powering BI and ML initiatives.

Bring 4+ years of data engineering or related experience, strong SQL/Python/Scala skills, and hands-on cloud data experience. Knowledge of Airflow, dbt, and big data frameworks is highly valued in a fast-paced, mission-driven environment.

Competenze

• 4+ years of experience in data engineering, software engineering, or database management.
• Proficiency in SQL, Python, or Scala for data processing and automation.
• Hands-on experience with cloud-based data solutions (AWS Redshift, Google BigQuery, Azure Synapse, Snowflake).
• Experience building ETL pipelines with tools such as Apache Airflow, dbt, Talend, or Fivetran.
• Strong understanding of data modeling, schema design, and database optimization.
• Experience with big data frameworks (Apache Spark, Hadoop, Kafka, Flink) is a plus.
• Familiarity with orchestration tools, containerization (Docker, Kubernetes), and CI/CD workflows.
• Knowledge of data security, governance, and compliance (GDPR, CCPA, SOC 2).
• Strong problem-solving and debugging skills with the ability to handle large-scale data challenges.
• Experience working in fast-paced, data-driven environments with cross-functional teams.

Mansioni

• Design, build, and maintain scalable data pipelines and ETL workflows to support analytics and reporting.
• Develop and optimize data warehouses and data lakes using cloud platforms such as AWS, Google Cloud, or Azure.
• Implement real-time and batch data processing solutions for various business needs.
• Work with structured and unstructured data, ensuring proper data modeling and storage strategies.
• Ensure data reliability, consistency, and scalability through best practices in architecture and engineering.
• Collaborate with data analysts, scientists, and software engineers to enable efficient data access and analysis.
• Automate data ingestion, transformation, and validation processes to improve data quality.
• Monitor and optimize query performance and data processing efficiency.
• Implement security, compliance, and governance standards for data storage and access control.
• Stay up to date with emerging data engineering trends, tools, and technologies.

Conoscenze

SQL

Python

Scala

Cloud-based data solutions

ETL pipelines

Airflow

dbt

Data modeling

Big data frameworks

Docker

Kubernetes

GDPR/CCPA/SOC 2

Strumenti

AWS Redshift

Google BigQuery

Azure Synapse

Snowflake

Apache Airflow

dbt

Talend

Fivetran

Apache Spark

Hadoop

Kafka

Flink

Docker

Kubernetes

Descrizione del lavoro

Air Apps is seeking a Data Engineer to design, build, and optimize data pipelines, warehouses, and lakes. You’ll work with analysts, scientists, and engineers to create scalable data infrastructure powering BI and ML initiatives.

Bring 4+ years of data engineering or related experience, strong SQL/Python/Scala skills, and hands-on cloud data experience. Knowledge of Airflow, dbt, and big data frameworks is highly valued in a fast-paced, mission-driven environment.