Cloud Data Platform Engineering Manager

Jobtailor; California (MO); On-site; USD 120,000 - 160,000; Full time

Job summary

Jobtailor is seeking a seasoned Data Engineer to build and maintain cloud-native data pipelines for batch and streaming ingestion. You will develop analytics-ready data transformations and modeling patterns supporting AI/ML use cases, and operate robust CI/CD driven workflows.

The role emphasizes data quality, observability, secure access, and cost optimization across cloud platforms, collaborating with cross-disciplinary teams to deliver production-ready data products.

Qualifications

• Bachelor's degree in a relevant field.
• 4+ years of experience in data engineering or similar roles.
• Strong proficiency in Python and SQL for data processing and testing.
• Hands-on experience with cloud platforms (AWS, GCP, Azure) and cloud-native storage/compute services.
• Experience with modern data processing or lakehouse technologies (Spark, Databricks, Snowflake, Redshift, Delta Lake, Iceberg).
• Experience with Airflow and CI/CD using Git-based workflows.
• Solid understanding of data modeling, data integration, and software engineering best practices.

Responsibilities

• Develop and maintain cloud-native data pipelines for batch and streaming ingestion.
• Develop transformation and modeling patterns enabling analytics and AI/ML use cases.
• Build and operate orchestration workflows with versioning, testing, promotion, rollback, and runbooks.
• Embed data quality, lineage, and observability into pipelines and datasets.
• Implement secure data access patterns aligned to enterprise and regulatory requirements.
• Optimize performance and cost across compute and storage.
• Collaborate with scientists, analysts, data scientists, and engineers to translate requirements into production solutions.
• Contribute to reusable standards, templates, documentation, examples, and onboarding materials.
• Stay current with technologies and best practices; support skill development and knowledge sharing.
• Pair with domain teams to define datasets and data products.
• Review pull requests and improve data quality checks and observability dashboards.
• Troubleshoot production issues, maintain runbooks, and perform root-cause follow-up.
• Ship changes through CI/CD and document delivered solutions.

Skills

Python Proficiency

SQL Proficiency

Airflow (Workflow)

Data Pipeline Development

Data Modeling

CI/CD

Data Quality

Observability

Root Cause Analysis

Education

Bachelor's degree

Tools

AWS

GCP

Azure

Spark

Databricks

Snowflake

Redshift

Delta Lake

Iceberg

Git

Job description

Jobtailor is seeking a seasoned Data Engineer to build and maintain cloud-native data pipelines for batch and streaming ingestion. You will develop analytics-ready data transformations and modeling patterns supporting AI/ML use cases, and operate robust CI/CD driven workflows.

The role emphasizes data quality, observability, secure access, and cost optimization across cloud platforms, collaborating with cross-disciplinary teams to deliver production-ready data products.