Principal Engineer I - Senior Data Engineer

Software Resources, Inc.; Phoenix (AZ); Hybrid; USD 120,000 - 160,000; Full time

Benefits offered by this job

Medical, dental, and vision coverage

401(k) with company match

Short-term disability

Life insurance with AD&D

Job summary

Software Resources, Inc. in Phoenix, AZ, seeks a senior Data Engineer to design and implement an enterprise data platform for Bank Regulatory Reporting. You will develop end-to-end data solutions and build robust ETL pipelines using PySpark and Azure data services.

Strong focus on performance tuning, TDD, and collaboration with governance and architecture teams. Hybrid schedule with on-site work in Phoenix and occasional travel as needed.

Qualifications

• 5+ years of data engineering experience.
• Expert in PySpark.
• 8+ years data engineering experience in ETL concepts with Azure Data Factory, Synapse, and/or Databricks.
• 5+ years designing and supporting cloud data solutions with Azure Data Lake, Data Factory, and Synapse.
• Bachelor’s degree in engineering or related field.
• Expert RDBMS and query language (T-SQL, PL/SQL, Spark SQL).
• Design and build reusable, dynamic PySpark components and frameworks.
• Support migration from Azure Synapse to Microsoft Fabric.

Responsibilities

• Design, build, and implement critical components of an enterprise data platform.
• Develop end-to-end data engineering solutions across the data lifecycle.
• Build ETL and data pipelines for creation, transformation, storage, and sharing.
• Write dynamic, parameterized PySpark code.
• Create shared frameworks and notebook structures.
• Perform performance tuning and optimization for Spark workloads.
• Implement Azure DevOps CI/CD pipelines per Bank standards.
• Apply TDD to data solutions and mentor team members.

Skills

PySpark

Azure Data Factory

Azure Synapse Analytics

Databricks

Azure DevOps

T-SQL

PL/SQL

Spark SQL

Expert RDBMS

Architectural design

Education

Bachelor’s degree in engineering

Tools

Visio

Archimate

Lucidchart

SAS

Tableau

PowerBI

Job description

Join a hybrid team to build and modernize an enterprise data platform supporting the Bank Regulatory Reporting Program.

• Design, build, and implement critical components of an enterprise data platform aligned to strategic data and analytics needs for the Bank Regulatory Reporting Program
• Develop end-to-end data engineering solutions across the full data lifecycle, including efficient technical hygiene and implementation using Azure DevOps for the Bank Regulatory Reporting and LFI strategy/roadmap
• Build ETL and data pipeline capabilities covering creation, transformation, storage, archiving, analysis, and sharing across the Bank and partner systems
• Write dynamic, parameterized, and reusable PySpark code
• Create shared frameworks including helper functions and companion notebook structures
• Perform performance tuning and optimization for Spark workloads
• Implement Azure DevOps CI/CD pipelines for data solutions in accordance with Bank technical standards
• Apply Test Driven Development (TDD) to data solutions designed, built, and implemented
• Lead implementation of outcomes, recommendations, and designs from Data Governance and Enterprise Architecture
• Conduct data exploration and profiling using tools such as SSAS
• Participate in data architecture decisions
• Provide technical mentorship to team members and other data professionals
• Collaborate with product owners and business stakeholders to understand business requirements and operational processes
• Implement quality and compliance by design in all data solutions
• Include support for unstructured data and big data methods

Key skills & requirements

• 5+ years of experience (minimum)
• Expert in PySpark (required)
• 8+ years data engineering experience in ETL concepts and processes, including enterprise data warehouse capabilities and database principles; preferably strong to expert in Azure Data Factory, Azure Synapse Analytics, and/or Databricks
• 5+ years designing, implementing, and supporting cloud data solutions; Azure Data Lake, Azure Data Factory, Azure Data Services, Azure Synapse, Azure Logic Apps, and Azure DevOps experience strongly preferred
• Bachelor’s degree in engineering or related field
• Expert RDBMS and query language experience with at least one of: T-SQL, PL/SQL, Spark SQL
• Ability to design and build reusable, dynamic PySpark components including companion notebook frameworks, shared helper functions/libraries, and parameter-driven ingestion and transformation patterns
• Support and influence migration from Azure Synapse to Microsoft Fabric, including modernization of patterns and frameworks
• Expert conceptual, logical, and physical data design
• Azure certifications such as Azure Fundamentals, Azure Fabric Data Engineer, Azure Data Scientist, and/or Azure DevOps Engineer
• Experience using design tools for conceptual architecture diagrams and data flow diagrams such as Visio, Archimate, Lucidchart
• Excellent communication skills verbal and written

Tools & technologies

• PySpark, Azure DevOps, ETL
• Azure Data Factory, Azure Synapse Analytics, Databricks
• Azure Data Lake, Azure Data Services, Azure Logic Apps, Microsoft Fabric
• SSAS
• T-SQL, PL/SQL, Spark SQL
• Visio, Archimate, Lucidchart
• SAS, Tableau, PowerBI

Location & work schedule

• Phoenix, AZ (hybrid)
• 4 days on site and Fridays remote in Phoenix, AZ or Dallas, TX

Benefits

• Medical, dental, and vision coverage
• 401(k) with company match
• Short-term disability
• Life insurance with AD&D

Preferred experience

• Experience in Agile, SAFe, and/or Scrum
• Experience integrating with data quality, data catalog, and data lineage tools
• Additional cloud data certifications with major providers (Azure, AWS, GCP)
• Banking/financial services or other highly regulated industry experience
• Regulatory Reporting experience, including Large Financial Institution requirements