Senior Software Engineer II (Data Pipelines)

twentysix; El Segundo (CA); On-site; USD 150,000 - 230,000; Full time

Job summary

twentysix is seeking a seasoned data engineering leader to drive automation and reliability of data pipelines. You will own Airflow-based DAG orchestration, manage a portfolio of dbt models, and lead best practices in testing, CI/CD, and deployment on Google Cloud Platform.

You will mentor engineers, contribute to architecture decisions, and champion scalable patterns while ensuring data governance and security across systems.

Qualifications

• 7+ years of software development experience.
• Expertise in Python, SQL and data pipelines.
• Experience building and automating ETL/ELT pipelines using Airflow and DAG-based workflow management.
• dbt models with macros and tests.
• Deep experience with PostgreSQL and BigQuery.
• Familiarity with Looker or similar BI tools.
• Experience with Google Cloud Platform, Kubernetes, and IaC with Terraform.
• CI/CD and test-driven development.

Responsibilities

• Act as a technical leader and SME in data pipeline automation and workflow orchestration.
• Design, implement, and maintain complex data solutions emphasizing automation with Airflow, dbt, and Google Cloud data products.
• Manage a large portfolio of dbt models using macros and DRY patterns.
• Advocate for test-driven development and support QA in CI/CD processes.
• Monitor, troubleshoot, and optimize data pipelines and workflows.
• Architect reusable patterns that scale with business needs.
• Provide implementation, configuration, and deployment documentation.
• Mentor engineers and promote technical excellence.
• Stay current with industry trends to drive continuous improvement.

Skills

Leadership

Mentoring

Agile methodologies

CI/CD

Problem solving

Tools

Python

SQL

Airflow

dbt

PostgreSQL

BigQuery

Looker

Google Cloud Platform

Kubernetes

Terraform

Prometheus

Grafana

GitHub Actions

Job description

You Will Be

• Acting as a technical leader and Subject Matter Expert (SME) in data pipeline automation and workflow orchestration.

• Designing, implementing, and maintaining complex, reliable, data solutions with a focus on automation using Airflow, dbt, and Google Cloud data products.

• Managing a large portfolio of dbt models, leveraging macros and DRY patterns.

• Advocating for test-driven development and assisting QA in developing a robust and reliable process for continuous integration and delivery.

• Monitoring, troubleshooting, and optimizing the performance of data pipelines and workflows.

• Architecting solutions and reusable patterns that scale with business needs.

• Providing implementation, configuration, and deployment documentation.

• Proactively addressing issues and problems, generating and implementing innovative solutions.

• Participating in all agile ceremonies, including daily standups and regular sprint planning.

• Mentoring other engineers and fostering a culture of technical excellence.

• Staying up-to-date with the latest industry trends and technologies to drive continuous improvement and innovation in data engineering practices.

• Ensuring data security, governance, and compliance with relevant standards and privacy restrictions.

You Must Have

Minimum 7+ years of experience in software development, with extensive experience in Python, SQL, and data pipelines.

Expertise in building and automating ETL/ELT pipelines using Airflow and DAG-based workflow management software. Airbyte for data ingestion is a plus, but not required.

Deep experience managing dbt models using macros and dbt tests.Mastery of Python or comparable scripting language, API integrations, and software architecture.

Deep experience with databases like PostgreSQL and BigQuery, including query optimization for performance and cost.

Good understanding of business intelligence tools like Looker or comparable alternatives.Experience with Google Cloud Platform, Kubernetes, and managing infrastructure as code using Terraform.

Proficiency with advanced data formats (Parquet, Avro, Hive, JSONL) and data integration techniques.

Experience with monitoring and logging tools (e.g., Prometheus, Grafana) is a plus.

Familiarity with version control systems and CI/CD tools like GitHub Actions.

Strong command of agile methodologies, continuous integration, and test-driven development.

Exceptional problem-solving skills and technical leadership.

Ability to influence and guide cross-functional teams and projects.