Azure Data Engineer: PySpark & Databricks Expert

TechDigital Group; Minneapolis (MN); On-site; USD 90,000 - 150,000; Full time

Job summary

An established industry player is seeking an experienced Azure Data Engineer to design and implement cutting-edge cloud solutions. In this dynamic role, you will leverage your expertise in Azure services, including Data Lake Storage and Data Factory, to develop robust data pipelines and optimize data workflows. Collaborating with cross-functional teams, you will ensure the reliability and performance of data processes while documenting technical designs and best practices. If you are passionate about cloud technologies and thrive in a fast-paced environment, this opportunity offers a chance to make a significant impact in the field of data engineering.

Qualifications

• 10+ years of experience with Azure cloud services and data engineering.
• Strong proficiency in PySpark and experience with Databricks.

Responsibilities

• Design and manage Azure cloud solutions for various projects.
• Develop data pipelines using Databricks for data ingestion and analysis.

Skills

Snowflake

Azure Databricks

Azure Data Factory

Pyspark

Python

SQL

Education

Bachelor's degree in Computer Science

Bachelor's degree in Engineering

Related field

Tools

Git

Job description

What are the top 3 skills required for this role?
1. z/OS Architecture & Sysplex Integration
2. Mainframe Networking (SNA, VTAM, TCP/IP, OSA, OSPF)
3. Mainframe Infrastructure & Performance Engineering

Job Description/ Responsibilities
Mainframe Networking & Infrastructure:
• Manage and support Mainframe Networking Protocols: SNA, VTAM, TCP/IP
• Configure and maintain OSA (Open System Adapter) networking interfaces
• Strong understanding of Enterprise Network Infrastructure and Security
• Knowledge of routing protocols such as Open Shortest Path First (OSPF)
• Manage z/OS network communications, connectivity, and performance optimization
System Programming & Architecture Skills:
• Strong experience in z/OS System Programming and System z architecture
• Expertise in Sysplex configuration, LPAR management, and system integration
• Hands-on with SMP/E, IOCP, HCD, PARMLIB, and system configurations
• Install and maintain z/OS releases, RSU maintenance, and ISV products
• Knowledge of JCL, utilities, batch and online processing environments
Key Responsibilities:
• Lead installation, configuration, and upgrading of z/OS environments across development and production systems
• Design and manage z/OS architecture, Sysplex integration, and CFRM policies
• Handle hardware and software upgrade projects, including datacenter migration and exit planning
• Perform performance analysis, tuning, capacity planning, and workload management (WLM)
• Monitor system software access, performance, and resource utilization
• Manage and maintain z/OS, JES3 environments and ISV software
• Work closely with application teams to ensure production stability and issue resolution
• Define and enforce security, access control, and software usage policies
• Implement and validate backup, recovery, and disaster recovery (DR) procedures
• Ensure high availability, system reliability, and environment stability
• Support incident/problem determination and resolution within SLA timelines
• Provide technical leadership, mentoring, and guidance to teams