Senior Data Engineer – AI-Powered Data Pipelines

Ingrain Systems Inc; San Francisco (CA); On-site; USD 180,000 - 260,000; Full time

Job summary

Ingrain Systems Inc. is seeking a Senior Data Engineer with deep data engineering expertise and hands-on AI skills in San Francisco, CA.

You will build scalable data infrastructure across ingestion, transformation, orchestration, and delivery while leveraging AI tooling as a core part of engineering and shipping. You will collaborate with backend, BI, and ML teams to design event-driven pipelines, maintain data quality, and participate in on-call reliability, agile processes, and modern tooling

Qualifications

• 5+ years of data engineering experience in large-scale data platforms.
• Strong Python and SQL with Spark experience (PySpark).
• Proficient in building scalable ETL/ELT pipelines and data orchestration.
• Experience with CI/CD and monitoring data workflows.

Responsibilities

• Design, build, and maintain scalable data ingestion and processing pipelines.
• Optimize ETL/ELT workflows and resolve production bottlenecks.
• Integrate multiple structured and unstructured data sources.
• Design event-driven architectures using Kafka, EventBridge, or SNS/SQS.
• Develop and maintain schema, validation, and backward compatibility.
• Collaborate with backend, BI, and ML teams to deliver data products.
• Implement CI/CD pipelines for data workflows and tests.

Skills

Python

SQL

PySpark/Scala

Unix scripting

Airflow

Kafka

DBT

Git/Jenkins/GitHub Actions

Docker

APIs

Tools

Airflow

SnapLogic

DBT

Git

Jenkins

Docker

Job description

Ingrain Systems Inc. is seeking a Senior Data Engineer with deep data engineering expertise and hands-on AI skills in San Francisco, CA.

You will build scalable data infrastructure across ingestion, transformation, orchestration, and delivery while leveraging AI tooling as a core part of engineering and shipping. You will collaborate with backend, BI, and ML teams to design event-driven pipelines, maintain data quality, and participate in on-call reliability, agile processes, and modern tooling