Solution Engineer, Data Engineering – Manager

Jobtailor; California (MO); On-site; USD 120,000 - 160,000; Full time

Job summary

Jobtailor is seeking a seasoned Data Engineer to build and maintain cloud-native data pipelines for batch and streaming ingestion. You will develop analytics-ready data transformations and modeling patterns supporting AI/ML use cases, and operate robust CI/CD driven workflows.

The role emphasizes data quality, observability, secure access, and cost optimization across cloud platforms, collaborating with cross-disciplinary teams to deliver production-ready data products.

Qualifications

• Bachelor's degree in a relevant field.
• 4+ years of experience in data engineering or similar roles.
• Strong proficiency in Python and SQL for data processing and testing.
• Hands-on experience with cloud platforms (AWS, GCP, Azure) and cloud-native storage/compute services.
• Experience with modern data processing or lakehouse technologies (Spark, Databricks, Snowflake, Redshift, Delta Lake, Iceberg).
• Experience with Airflow and CI/CD using Git-based workflows.
• Solid understanding of data modeling, data integration, and software engineering best practices.

Responsibilities

• Develop and maintain cloud-native data pipelines for batch and streaming ingestion.
• Develop transformation and modeling patterns enabling analytics and AI/ML use cases.
• Build and operate orchestration workflows with versioning, testing, promotion, rollback, and runbooks.
• Embed data quality, lineage, and observability into pipelines and datasets.
• Implement secure data access patterns aligned to enterprise and regulatory requirements.
• Optimize performance and cost across compute and storage.
• Collaborate with scientists, analysts, data scientists, and engineers to translate requirements into production solutions.
• Contribute to reusable standards, templates, documentation, examples, and onboarding materials.
• Stay current with technologies and best practices; support skill development and knowledge sharing.
• Pair with domain teams to define datasets and data products.
• Review pull requests and improve data quality checks and observability dashboards.
• Troubleshoot production issues, maintain runbooks, and perform root-cause follow-up.
• Ship changes through CI/CD and document delivered solutions.

Skills

Python Proficiency

SQL Proficiency

Airflow (Workflow)

Data Pipeline Development

Data Modeling

CI/CD

Data Quality

Observability

Root Cause Analysis

Education

Bachelor's degree

Tools

AWS

GCP

Azure

Spark

Databricks

Snowflake

Redshift

Delta Lake

Iceberg

Git

Job description

• Develop and maintain cloud-native data pipelines for batch and streaming ingestion


• Develop transformation and modeling patterns enabling analytics and AI/ML use cases


• Build and operate orchestration workflows with versioning, testing, promotion, rollback, and runbooks


• Embed data quality, lineage, and observability into pipelines and datasets


• Implement secure data access patterns aligned to enterprise and regulatory requirements


• Optimize performance and cost across compute and storage


• Collaborate with scientists, analysts, data scientists, and engineers to translate requirements into production solutions


• Contribute to reusable standards, templates, documentation, examples, and onboarding materials


• Stay current with technologies and best practices; support skill development and knowledge sharing


• Pair with domain teams to define datasets and data products


• Review pull requests and improve data quality checks and observability dashboards


• Troubleshoot production issues, maintain runbooks, and perform root-cause follow-up


• Ship changes through CI/CD and document delivered solutions


• Report to the Principal Engineer, Data Engineering

Requirements

• Bachelor's degree in a relevant field (e.g., Computer Science, Data Science, Data Engineering, Bioinformatics, Engineering, or related discipline)


• 4+ years of experience in data engineering, software engineering, or closely related roles


• Strong proficiency in Python and SQL for data processing, automation, and testing


• Hands-on experience with cloud platforms (AWS, GCP, Azure) and cloud-native storage/compute services


• Experience with one or more modern data processing or lakehouse/warehouse technologies (e.g., Spark, Databricks, Snowflake, Redshift, Delta Lake, Iceberg)


• Experience with workflow orchestration and scheduling (e.g., Airflow) and CI/CD using Git-based workflows


• Strong understanding of data modeling, data integration patterns, and software engineering best practices (code review, testing, documentation)


• Working knowledge of security best practices for data platforms (IAM, secrets management, encryption, least privilege) and operating in regulated environments


• Strong problem-solving skills and eagerness to learn in a collaborative environment


• Fluent in English; capable of clear technical communication across scientific and engineering disciplines


• Candidates must be authorized to be employed in the U.S. by any employer


• U.S. work visa sponsorship is not available for this role now or in the future


• This position requires permanent work authorization in the United States

Core Competencies

Demonstrates expertise in developing and maintaining cloud-native data pipelines, with strong proficiency in Python and SQL for data processing. Capable of implementing secure data access patterns and optimizing performance across cloud platforms while collaborating effectively with cross-functional teams.

Highest-signal resume keywords

• Cloud-Native Data Pipeline Development


• Python Proficiency


• SQL Proficiency


• Workflow Orchestration (Airflow)


• Data Modeling and Integration

ATS Optimization Keywords

Hard Skills

• Data Engineering


• Software Engineering


• Data Processing


• Automation


• Testing


• Data Integration Patterns


• CI/CD


• Data Quality Checks


• Observability


• Root-Cause Analysis

Soft Skills

• Problem-Solving


• Collaboration


• Technical Communication


• Eagerness to Learn

Industry Keywords

• Data Quality


• Data Lineage


• Observability


• Security Best Practices


• Regulated Environments

Tools & Technologies

• AWS


• GCP


• Azure


• Spark


• Databricks


• Snowflake


• Redshift


• Delta Lake


• Iceberg


• Git