Senior Data Engineer (PySpark) - Hybrid Azure Data Platform

Software Resources, Inc.; Phoenix (AZ); Hybrid; USD 120,000 - 160,000; Full time

Benefits offered by this job

Medical, dental, and vision coverage

401(k) with company match

Short-term disability

Life insurance with AD&D

Job summary

Software Resources, Inc. in Phoenix, AZ, seeks a senior Data Engineer to design and implement an enterprise data platform for Bank Regulatory Reporting. You will develop end-to-end data solutions and build robust ETL pipelines using PySpark and Azure data services.

Strong focus on performance tuning, TDD, and collaboration with governance and architecture teams. Hybrid schedule with on-site work in Phoenix and occasional travel as needed.

Qualifications

• 5+ years of data engineering experience.
• Expert in PySpark.
• 8+ years data engineering experience in ETL concepts with Azure Data Factory, Synapse, and/or Databricks.
• 5+ years designing and supporting cloud data solutions with Azure Data Lake, Data Factory, and Synapse.
• Bachelor’s degree in engineering or related field.
• Expert RDBMS and query language (T-SQL, PL/SQL, Spark SQL).
• Design and build reusable, dynamic PySpark components and frameworks.
• Support migration from Azure Synapse to Microsoft Fabric.

Responsibilities

• Design, build, and implement critical components of an enterprise data platform.
• Develop end-to-end data engineering solutions across the data lifecycle.
• Build ETL and data pipelines for creation, transformation, storage, and sharing.
• Write dynamic, parameterized PySpark code.
• Create shared frameworks and notebook structures.
• Perform performance tuning and optimization for Spark workloads.
• Implement Azure DevOps CI/CD pipelines per Bank standards.
• Apply TDD to data solutions and mentor team members.

Skills

PySpark

Azure Data Factory

Azure Synapse Analytics

Databricks

Azure DevOps

T-SQL

PL/SQL

Spark SQL

Expert RDBMS

Architectural design

Education

Bachelor’s degree in engineering

Tools

Visio

Archimate

Lucidchart

SAS

Tableau

PowerBI

Job description

Software Resources, Inc. in Phoenix, AZ, seeks a senior Data Engineer to design and implement an enterprise data platform for Bank Regulatory Reporting. You will develop end-to-end data solutions and build robust ETL pipelines using PySpark and Azure data services.

Strong focus on performance tuning, TDD, and collaboration with governance and architecture teams. Hybrid schedule with on-site work in Phoenix and occasional travel as needed.