Senior Data Engineer — End-to-End Data & ML Pipelines

Seamless.AI; Northern (KY); Hybrid; USD 140.000 - 200.000; Vollzeit

Zusammenfassung

Seamless delivers the world’s best sales leads through an advanced real-time contact and company search engine. The Senior Data Engineer will own critical data pipelines, collaborate on ML scoring models, and help scale our data infrastructure to millions of records across a hybrid environment.

You will drive architecture decisions, implement PySpark ETL jobs, and work with AWS services like Glue, EMR, S3, and Step Functions while ensuring data quality with automated tests and robust lineage

Qualifikationen

• 5+ years of professional data engineering experience with end-to-end ownership of projects.
• Comfortable defending architecture decisions with minimal oversight.
• Able to own a project from analysis through production and testing.
• Able to work with dependencies on other teams' data deliverables and adjust scope accordingly.
• Clear communicator who documents architecture decisions and tradeoffs.
• Deep, hands-on experience with Apache Spark / PySpark and understanding of its execution.
• Complex SQL fluency: multi-way joins, window functions, and large-scale aggregations.
• Experience with AWS data services: Glue, EMR, S3, Step Functions, Lambda, EventBridge, CloudWatch.
• Experience writing automated tests for data pipelines (pytest or similar).

Aufgaben

• Building and maintaining PySpark ETL jobs.
• Supporting ML-based scoring models that assess data quality (phone numbers, emails).
• Writing and optimizing SQL across large lake tables.
• Contributing to architecture decisions for scoring systems and ML positioning within score ranges.
• Writing unit and integration tests for data transformations and model inference code.
• Working within a medallion-style lake architecture (bronze/silver/gold).
• Identifying opportunities to leverage AI and agentic workflows for data acquisition and profile aggregation.

Kenntnisse

Apache Spark / PySpark

SQL

AWS data services

ETL orchestration

Data modeling

Unit/Integration testing

Communication

Tools

Airflow

dbt

Great Expectations

AWS Glue

Jobbeschreibung

Seamless delivers the world’s best sales leads through an advanced real-time contact and company search engine. The Senior Data Engineer will own critical data pipelines, collaborate on ML scoring models, and help scale our data infrastructure to millions of records across a hybrid environment.

You will drive architecture decisions, implement PySpark ETL jobs, and work with AWS services like Glue, EMR, S3, and Step Functions while ensuring data quality with automated tests and robust lineage