Sr. Data Engineer

PadSplit; United States; Remote; USD 140.000 - 195.000; Vollzeit

Zusammenfassung

PadSplit is expanding its analytics platform and seeks a hands-on Sr. Data Engineer to build and maintain ingestion and transformation pipelines across Dagster (or Airflow), dbt, and Snowflake.

You will work with Python, Airbyte, and AWS to ensure reliable daily operations from Postgres to Snowflake and third-party API loads. You’ll design and optimize dimensional models, implement incremental loads, and review PRs to keep pipelines healthy and observable.

Qualifikationen

• Knowledge of dimensional modeling (facts, dimensions, SCD) and data warehousing patterns.
• Hands-on experience with orchestration tools (Dagster or Airflow).
• Experience building and maintaining dbt models, tests, and docs.
• Proficient Python for large-scale data workflows; AWS familiarity.

Aufgaben

• PR-driven development of data pipelines and models across Dagster, Airflow, and dbt.
• Tuning and debugging pipelines, including REST/API syncs and large extracts.
• Configure or troubleshoot Airbyte connections and related data loads.
• Participate in code reviews, runbooks, and release prep.

Kenntnisse

Data modeling

Python programming

Incremental loads

Reliability mindset

Communication

Problem solving

Tools

Dagster

Airflow

dbt

Snowflake

Airbyte

Terraform

Jobbeschreibung

The Role We Need

PadSplit is growing its analytics platform and needs a hands‑on Sr. Data Engineer to work alongside our existing DE lead. This person will build and maintain ingestion and transformation pipelines across Dagster (or Airflow), dbt, and Snowflake, with supporting work in Python, Airbyte, and AWS. The role combines building new pipelines and data models with providing real coverage on critical paths — especially the daily Postgres → Snowflake → dbt flow and third‑party API loads — all shipped through reviewed pull requests rather than one‑off scripts.

The Person We Are Looking For

We’re looking for a practitioner who thinks natively in dimensions, facts, and slowly changing dimensions — someone who knows when to use a full refresh versus an incremental load and how that choice affects idempotency and backfills. This person writes clear, reviewable PRs and gives equally thoughtful reviews, with attention to scoped diffs, sensible tests, and failure modes. They’re comfortable in complex Python data flows and have enough AWS literacy to reason about task roles, buckets, and cross‑account access without needing to own all of platform engineering.

Here’s What You’ll Be Doing Day-to-Day

• PR-driven shipping: Opening and merging pull requests for new or updated Dagster jobs, assets, schedules, and sensors, plus dbt models, tests, and documentation.
• Infrastructure tweaks: Writing occasional Terraform for secrets, environment variables, or job sizing when a pipeline needs it.
• Pipeline implementation: Building and debugging Python pipelines covering REST/API syncs, large Postgres extracts, Parquet loads, and Snowflake COPY operations.
• Airbyte management: Configuring or troubleshooting Airbyte connections wherever managed sync is the right fit.
• Production monitoring: Watching production runs and investigating failures related to IAM, OOM, Spot instances, or bad watermarks.
• Backfills & catch‑ups: Running backfills and incremental catch‑ups with a clear story for what landed and why.
• Modeling partnership: Working with analytics and product on dim/fct/x_fct design, incremental strategies, and data quality.
• Code review & runbooks: Participating in code review, release prep, and writing short runbooks so others can operate your pipelines when you’re out.

Here’s What You’ll Need to Be Successful

• Warehouse fundamentals: Solid grasp of relational databases and warehouse patterns — keys, grain, normalization vs. star schema, and how SCD behavior gets encoded.
• Orchestration experience: Practical, hands‑on Dagster (or Airflow) experience — not just writing SQL inside a scheduler UI.
• dbt proficiency: Real experience building and maintaining models, tests, and documentation in dbt.
• Python at scale: Comfort reading and writing Python that moves data at scale across extract, transform, and load steps.
• AWS working knowledge: Practical familiarity with S3, IAM, and ECS/Fargate at a “debug my job” level.
• EL tool familiarity: Experience with Airbyte or similar extract‑and‑load tools.
• PR discipline: The discipline to write pull requests others can easily review, and to give equally rigorous reviews in return.
• Reliability mindset: A track record of keeping pipelines healthy and modeling consistent across full refresh and incremental paths, without becoming a single point of failure.

The Interview Process

• Your application will be reviewed for possible next steps by a real human being from the PeopleOps team.
• If you meet eligibility requirements, the next step would be a video interview with a member of the PeopleOps team for about thirty (30) minutes.
• If warranted, the next step would be a video interview with our Principal Data Scientist for forty‑five (45) minutes.
• If warranted, the next step would be a video panel interview with key stakeholders at PadSplit for one and a half (1.5) hours.
• If warranted, the next and final step would be a video interview with a key leader in the company for thirty (30) minutes.
• If warranted, we move to offer!

Compensation, Benefits, and Perks

• Fully remote position – we swear!
• Competitive compensation package including an equity incentive plan and company‑wide bonus opportunity
• National medical, dental, and vision healthcare plans
• Company provided life insurance policy
• Optional accidental insurances, FSA, and DCFSA benefits
• Unlimited paid‑time (PTO) policy with eleven (11) company‑observed holidays
• 401(k) plan
• Twelve (12) weeks of paid time off for both birth and non‑birth parents
• The opportunity to do what you love at a company that is at the forefront of solving the affordable housing crisis

Compensation

Compensation is based on the role’s scope, national market benchmarks, the person’s expertise and experience, and the impact of their contributions to our business goals.

Additional Information

Please note: Although the job posting says it’s in Atlanta, Georgia, this is a fully remote position. This is a result of our Applicant Tracking System requiring a location to post the role on LinkedIn.

Notice to Applicants

All employees must be able to establish their identity and authorization to work in the United States by completing Form I-9 after an offer of employment. For this role, we are unable to provide employment‑based visa sponsorship, now or in the future. Applicants must be authorized to work in the U.S. without requiring sponsorship from PadSplit. Candidates who are U.S. citizens, lawful permanent residents, refugees, asylees, and others with unrestricted work authorization are welcome to apply. PadSplit is an equal opportunity employer and does not discriminate on the basis of citizenship status or national origin.