Senior Software Engineer II, Data Pipelines

Jobtailor; El Segundo (CA); On-site; USD 140,000 - 190,000; Full time

Job summary

Jobtailor is seeking a senior data engineering lead in El Segundo, CA to own and advance our data pipeline automation and workflow orchestration. You will architect scalable solutions using Airflow, dbt, and Google Cloud, while mentoring engineers and improving CI/CD and testing practices.

You will collaborate with cross-functional teams, optimize performance and costs in BigQuery and PostgreSQL, and champion data governance and security standards in an agile environment.

Qualifications

• Minimum 7+ years of experience in software development.
• Experience building and automating ETL/ELT pipelines using Airflow and DAG-based workflow management software.
• Deep experience managing dbt models using macros and dbt tests.
• Strong proficiency in Python and SQL.
• Familiarity with data security, governance, and privacy standards.

Responsibilities

• Act as a technical leader and SME in data pipeline automation and workflow orchestration.
• Design, implement, and maintain reliable data solutions using Airflow, dbt, and Google Cloud data products.
• Mentor engineers and foster technical excellence.
• Monitor, troubleshoot, and optimize data pipeline performance.

Skills

Technical leadership

Mentoring

Agile methodologies

Problem-solving

Cross-functional influence

Tools

Airflow

dbt

PostgreSQL

BigQuery

Kubernetes

Terraform

GitHub Actions

Looker

GCP

Grafana

Prometheus

Job description

• Act as a technical leader and Subject Matter Expert in data pipeline automation and workflow orchestration
• Design, implement, and maintain reliable data solutions using Airflow, dbt, and Google Cloud data products
• Manage a large portfolio of dbt models using macros and DRY patterns
• Advocate for test-driven development and assist QA in developing a reliable continuous integration and delivery process
• Monitor, troubleshoot, and optimize data pipeline and workflow performance
• Architect scalable solutions and reusable patterns
• Provide implementation, configuration, and deployment documentation
• Proactively address issues and implement innovative solutions
• Participate in agile ceremonies, including daily standups and sprint planning
• Mentor other engineers and foster technical excellence
• Stay current with industry trends and technologies
• Ensure data security, governance, compliance, and privacy standards
• Report to the Director of IT

Requirements

• Minimum 7+ years of experience in software development
• Extensive experience in Python, SQL, and data pipelines
• Expertise building and automating ETL/ELT pipelines using Airflow and DAG-based workflow management software
• Experience with Airbyte for data ingestion is a plus, but not required
• Deep experience managing dbt models using macros and dbt tests
• Mastery of Python or a comparable scripting language, API integrations, and software architecture
• Deep experience with PostgreSQL and BigQuery, including query optimization for performance and cost
• Good understanding of business intelligence tools such as Looker or comparable alternatives
• Experience with Google Cloud Platform, Kubernetes, and Terraform for infrastructure as code
• Proficiency with Parquet, Avro, Hive, and JSONL data formats and data integration techniques
• Experience with Prometheus and Grafana is a plus
• Familiarity with version control systems and CI/CD tools such as GitHub Actions
• Strong command of agile methodologies, continuous integration, and test-driven development
• Exceptional problem-solving skills and technical leadership
• Ability to influence and guide cross-functional teams and projects
• Must be authorized to work in the U.S.; this position is not eligible for immigration sponsorship

Core Competencies

Demonstrates expertise in data pipeline automation and workflow orchestration, with a strong focus on designing and implementing scalable data solutions using Airflow, dbt, and Google Cloud products. Proficient in managing dbt models and advocating for best practices in test-driven development and continuous integration.

Highest-signal resume keywords

• Data Pipeline Automation
• Airflow and DAG-Based Workflow Management
• Python and SQL Proficiency
• Google Cloud Platform Experience
• Technical Leadership and Mentoring

ATS Optimization Keywords

Hard Skills

• Python
• SQL
• ETL/ELT Pipeline Automation
• Airflow
• Dbt
• PostgreSQL
• BigQuery
• Kubernetes
• Terraform
• Data Integration Techniques

Soft Skills

• Problem-Solving
• Technical Leadership
• Influencing Cross-Functional Teams

Industry Keywords

• Data Security
• Governance
• Compliance
• Agile Methodologies
• Continuous Integration

Tools & Technologies

• Google Cloud Data Products
• Looker
• Prometheus
• Grafana
• GitHub Actions