Senior Data Pipeline Architect for AI-Driven Platforms

BNY; Pittsburgh (Allegheny County); On-site; USD 120,000 - 180,000; Full time

Job summary

BNY is seeking a Data Pipeline Engineer to join the Data Innovation team in Pittsburgh, PA or Lake Mary, FL. You will design and build production-ready data pipelines powering IDS and knowledge graph initiatives, collaborating across platform, product and data teams to enable analytics and client solutions.

The role requires hands-on Python, Spark and SQL expertise, experience with ETL/ELT frameworks, and a track record of delivering scalable pipelines with robust data quality and lineage.

Qualifications

• Bachelor's degree or equivalent; advanced degree preferred in statistics or analytics.
• At least six years total work experience with emphasis on data analysis and BI.
• Extensive experience in data engineering and building scalable pipelines.
• Proficient in Python, Spark and SQL with ETL/ELT and orchestration tooling.
• Experience designing high-volume, resilient batch/streaming pipelines.
• Familiarity with Snowflake, AWS, Databricks and API-based data integration.

Responsibilities

• Design and build scalable data pipelines ingesting data from multiple sources.
• Transform diverse data formats into standardized time-series and event datasets.
• Develop reusable frames to normalize identifiers, units, and event data.
• Collaborate to map source data to canonical entities for graph ingestion.
• Implement data quality controls with lineage, provenance, and traceability.
• Enable multi-vendor ingestion, comparison, and quality analytics.
• Build modular cloud-native pipelines optimized for scale, performance and cost.

Skills

Python

Spark

SQL

ETL/ELT frameworks

Orchestration tools

Data modeling

High-volume pipelines

Knowledge graph / ontology

Education

Bachelor's degree or equivalent

Advanced degree preferred (statistics/analyze)

Tools

Snowflake

AWS

Databricks

Job description

BNY is seeking a Data Pipeline Engineer to join the Data Innovation team in Pittsburgh, PA or Lake Mary, FL. You will design and build production-ready data pipelines powering IDS and knowledge graph initiatives, collaborating across platform, product and data teams to enable analytics and client solutions.

The role requires hands-on Python, Spark and SQL expertise, experience with ETL/ELT frameworks, and a track record of delivering scalable pipelines with robust data quality and lineage.