Real-Time Data Engineer for Self-Service Analytics

Jobtailor; Burbank (CA); On-site; USD 120,000 - 180,000; Full time

Job summary

Jobtailor in California is seeking a data-focused engineer to build scalable data products and real-time data platforms. You will design and maintain batch and streaming pipelines, enabling self-service analytics through governed data models and robust data access patterns.

The role emphasizes collaboration with Product, Analytics, BI, and Engineering teams, as well as implementing observability and governance best practices for high-velocity environments.

Qualifications

• 2–4+ years of experience building and scaling ETL/ELT pipelines in production environments.
• Proven experience with workflow orchestration tools such as Airflow, Composer, or similar platforms.
• Working knowledge of distributed data processing concepts.
• Expert-level SQL skills for large-scale transformation and analytics.
• Experience designing scalable warehouse schemas and ML-ready data layers.
• Proven experience optimizing complex queries across multi-terabyte datasets.
• Proficiency in Python (or similar language) for data processing and ML pipeline integration.
• Experience with distributed processing frameworks such as Spark.
• Experience integrating data pipelines with ML platforms such as Vertex AI (preferred), Databricks ML, or equivalent.
• Experience building real-time data pipelines using Kafka, Pub/Sub, or similar technologies.
• Knowledge of feature streaming, low-latency data processing, and event-driven architectures.
• Excellent problem-solving skills with the ability to work in dynamic, high-velocity environments.
• Bachelor's or Master's degree in Computer Science, Engineering, or a related field (or equivalent experience)

Responsibilities

• Help build scalable data products, semantic layers, and real-time data platforms
• Develop solutions that power BI, analytics, experimentation, AI applications, agents, and conversational analytics experiences
• Design, develop, and maintain scalable batch (ETL/ELT) and near real-time streaming data pipelines
• Enable self-service analytics through governed and reusable business data models
• Implement monitoring, observability, and operational best practices
• Develop governed data access patterns for AI, conversational analytics, and MCP-based applications
• Build AI-ready data products that support machine learning, GenAI, AI agents, and chatbot applications
• Partner with Product, Analytics, BI, and Engineering stakeholders to deliver trusted data solutions
• Design scalable data models optimized for analytics, real-time reporting, and AI use cases
• Drive best practices for data quality, governance, metadata, and discoverability

Skills

ETL/ELT pipelines

Distributed processing concepts

SQL

Python

Real-time data pipelines

Data modeling

ML-ready data models

Query optimization

Problem solving

Education

Bachelor's or Master's degree in Computer Science, Engineering, or a related field

Tools

Airflow

Composer

Spark

Vertex AI

Databricks ML

Kafka

Pub/Sub

Job description

Jobtailor in California is seeking a data-focused engineer to build scalable data products and real-time data platforms. You will design and maintain batch and streaming pipelines, enabling self-service analytics through governed data models and robust data access patterns.

The role emphasizes collaboration with Product, Analytics, BI, and Engineering teams, as well as implementing observability and governance best practices for high-velocity environments.