Data Engineer for Real-Time AI Pipelines

HeyGen; Palo Alto, San Francisco, Los Angeles (CA, CA, CA); On-site; USD 180,000 - 220,000; Full time

Benefits offered by this job

Competitive salary and benefits

Dynamic work environment

Opportunities for professional growth

Collaborative culture

Access to innovative technologies

Job summary

A technology company specializing in visual storytelling seeks a Software Engineer in Palo Alto, CA. This role involves building and maintaining data pipelines and data infrastructures essential for innovative AI features. Candidates should have a degree in Computer Science or related field, 3-5+ years as a Backend Engineer, and strong skills in Python and SQL. The position offers a competitive salary, benefits, and a dynamic work environment focused on innovation.

Qualifications

• 3-5+ years of experience as a Backend Software Engineer with data processing responsibilities.
• Strong proficiency in Python for ETL/scripting and SQL for data modeling.
• Proactive, 'owner' mindset with the ability to work in a fast-paced environment.

Responsibilities

• Design and maintain robust batch and real-time data pipelines.
• Collaborate with ML engineers for data structures and APIs.
• Architect and manage data lakehouse solutions.
• Implement data quality checks and monitoring.
• Transform raw data into actionable products.

Skills

Python

SQL

AWS

GCP

Kafka

Spark

Computer Vision

Generative AI processing

Education

Bachelor’s/Master’s degree in Computer Science or related field

Tools

Snowflake

Databricks

Apache Iceberg

Job description

A technology company specializing in visual storytelling seeks a Software Engineer in Palo Alto, CA. This role involves building and maintaining data pipelines and data infrastructures essential for innovative AI features. Candidates should have a degree in Computer Science or related field, 3-5+ years as a Backend Engineer, and strong skills in Python and SQL. The position offers a competitive salary, benefits, and a dynamic work environment focused on innovation.