Data Engineer II

Essentia Health; Duluth (MN); Remote; USD 83,000 - 126,000; Full time

Job summary

An established industry player is seeking a Data Engineer II to design and optimize complex data pipelines and workflows. This role involves implementing advanced data processing techniques using frameworks like Apache Spark and Airflow, ensuring data quality and governance through automated solutions. You will collaborate with cross-functional teams to support data-driven decision-making and mentor junior engineers. This position offers a chance to work in a dynamic environment focused on innovation and efficiency, providing valuable insights for business-critical tools. Join a forward-thinking organization that values professional development and work-life balance.

Benefits

Medical, dental, and vision insurance

401(k) retirement savings plan

Tuition reimbursement

Flexible scheduling

Employee wellness programs

Qualifications

• 2-5 years of experience in data engineering or a related role.
• Proficiency in SQL and experience with database management systems.
• Strong programming skills in Python, Java, or Scala.

Responsibilities

• Design and optimize scalable data pipelines and workflows.
• Implement advanced data processing techniques using Apache Spark and Airflow.
• Collaborate with data scientists and analysts to understand data requirements.

Skills

SQL

Python

Apache Spark

Apache Airflow

Data Governance

Data Quality

Collaboration

Problem-solving

Education

Bachelor's degree in Computer Science

Bachelor's degree in Data Analytics

Bachelor's degree in Statistics

Tools

AWS Glue

Data Lake

Lakehouse

Data Catalogs

Job description

Building Location: Peerless Building Department: 1007670 ANALYTICS AND ARCHITECTURE - EH SS Job Description: The Data Engineer II is responsible for designing and optimizing scalable and complex data pipelines and workflows to handle large datasets. They are expected to implement advanced data processing and scheduling techniques using frameworks like Apache spark, ADF and airflow. Ensures data quality, governance, and security through automated testing and monitoring solutions. Collaborate with cross-functional teams to support data-driven decision-making and ensure the scalability and efficiency of data systems. Education Qualifications:

Key Responsibilities:

• Designs and optimizes scalable and robust data pipelines and workflows to handle large datasets to meet functional and non-functional business requirement
• Implements advanced data processing and scheduling techniques using frameworks like Apache Spark, ADF and Airflow
• Write and optimize advanced SQL queries for reporting and applications
• Ensures data quality, governance, and security through automated testing and monitoring solutions
• Work closely with data scientists, analysts, and other stakeholders to understand data requirements
• Provide technical guidance and mentorship to junior data engineers
• Develop infrastructure to support real-time and batch processing workflows
• Deliver actionable data products and insights for business-critical tools

Education Requirements:

• Bachelor’s degree in computer science, data analytics, statistics, or related field

Required Qualifications:

• 2-5 years of experience in data engineering or a related role
• Proficiency in SQL and experience with database management systems
• Strong programming skills in languages such as Python, Java, or Scala
• Experience with pipelines using orchestration tools such as Apache Airflow or AWS Glue for seamless data ingestion and delivery
• Experience with modern data management technologies (e.g., data lake, lakehouse, data catalogs)
• Experience supporting both batch and streaming data, enabling near real-time analytics and reporting

Preferred Qualifications:

• Familiarity with applying AI and machine learning techniques to data use cases
• Experience creating robust data transformation workflows, including metadata management, dependency tracking, and workload orchestration
• Excellent problem-solving skills and attention to detail
• Strong communication and collaboration skills

Licensure/CertificationQualifications: FTE: 1

Possible Remote/Hybrid Option:

Remote Shift Rotation: Day Rotation (United States of America) Shift Start Time: 8:00 AM Shift End Time: 4:30 PM Weekends: Holidays: No Call Obligation: No Union: Union Posting Deadline:

Compensation Range:

$83,428.80 - $125,153.60

Employee Benefits at Essentia Health*:

• Health Coverage: Medical, dental, vision, life and disability insurance, plus supplemental health benefit options to ensure employees' well-being.

• Retirement Savings Plans: 401(k) with employer contributions to support long-term financial security.

• Professional Development: Opportunities for career growth through training, tuition reimbursement, and educational programs.

• Work-Life Balance: Flexible scheduling, time off, holidays, and personal leave to help employees manage their professional and personal lives.

• Employee Wellness Programs: Initiatives focused on physical, mental, and emotional health, including fitness memberships, counseling services, and wellness activities.

*Eligibility for Essentia Health’s benefit programs vary. Please refer to the benefit summary provided to you, or contact our HR Service Center at [phone] for more information.