Staff Data Platform Engineer

Arch Systems; United States; On-site; USD 149,000 - 194,000; Full time

Job summary

Plume’s Data Services team seeks engineers to build and operate data pipelines powering our analytics. You’ll tackle large-scale data challenges with modern tools, delivering reliable data products and insightful metrics across our platform.

You’ll own pipeline design, implement security controls with Lake Formation and IAM, and mentor teammates as the team expands. Join us to evolve a hardware-independent, cloud-controlled experience platform for ISPs and their customers.

Qualifications

• 5+ years of software development experience building, scaling, and supporting production data pipelines.
• Hands-on expertise with AWS big data services particularly Athena for distributed querying and Lake Formation for security, cataloging, and access management.
• Solid Apache Airflow experience: writing DAGs, managing task dependencies, and debugging runtime failures.
• A DataOps mindset comfortable with CI/CD for pipelines, environment monitoring, IAM/permissions management, and owning what you ship.

Responsibilities

• Partner with stakeholders to define data requirements and translate them into robust pipeline designs.
• Build and maintain high-quality data pipelines with strong reliability and data integrity standards.
• Own data warehouse schema documentation and keep it current as systems evolve.
• Write clean, well-structured code and contribute to a culture of rigorous peer review.
• Continuously refactor and optimize pipelines for performance, cost efficiency, and operational simplicity.
• Triage and resolve production issues related to data quality and availability.
• Mentor junior engineers and new hires, helping them ramp up quickly and contribute confidently.
• Uphold data governance requirements including access controls, retention policies, residency rules, and de-identification.
• Contribute to the technology roadmap and drive best practice adoption across the team.
• Orchestrate complex workflows in Apache Airflow writing DAGs, managing dependencies, and building in automated error handling.
• Implement granular access controls and data security policies using AWS Lake Formation.
• Integrate automated data quality checks into deployment pipelines so issues are caught before they reach production

Skills

AWS Big Data

Apache Airflow

Python

SQL

Java/Scala

Apache Spark

Data Warehousing

Data Governance

CI/CD for pipelines

IAM/Permissions

DataOps mindset

Tools

Athena

Lake Formation

Apache Airflow

Job description

Plume’s Data Services team seeks engineers to build and operate data pipelines powering our analytics. You’ll tackle large-scale data challenges with modern tools, delivering reliable data products and insightful metrics across our platform.

You’ll own pipeline design, implement security controls with Lake Formation and IAM, and mentor teammates as the team expands. Join us to evolve a hardware-independent, cloud-controlled experience platform for ISPs and their customers.