Data Engineer

SFE; Chicago (IL); On-site; USD 100,000 - 140,000; Full time

Job summary

SFE is seeking an experienced Data Engineer to design, develop, and maintain scalable data pipelines and data platforms in an onsite, multi-city environment including Chicago IL, St. Louis MO and Richardson TX.

The ideal candidate will have hands-on experience with Databricks, Azure, Apache Spark, and strong Python/Scala programming skills to build robust data processing solutions for healthcare-related datasets.

Qualifications

• Hands-on experience with Databricks and Spark in a data engineering role.
• Strong knowledge of Microsoft Azure cloud services for data ingestion, storage, processing, and integration.
• Experience building scalable data pipelines and ETL/ELT workflows in large-scale environments.

Responsibilities

• Design and develop scalable data pipelines and ETL/ELT workflows using Databricks and Apache Spark.
• Develop data engineering solutions using Python and Scala.
• Work with Azure cloud services for data ingestion, storage, processing, and integration.
• Build and optimize Spark-based data processing applications for large-scale datasets.
• Develop data transformation, cleansing, and validation processes.
• Collaborate with data architects, analysts, and application teams to deliver reliable data solutions.
• Troubleshoot performance, data quality, and pipeline issues.
• Follow best practices for data security, scalability, monitoring, and deployment.

Skills

Databricks

Azure

Apache Spark

Python

Scala

ETL/ELT

Data pipelines

Distributed computing

Cloud data

Communication

Job description

Position: Data Engineer (Healthcare Exp Is Must)

Location: St. Louis MO Richardson TX Chicago IL (Onsite)

Duration: Contract

Job Description

We are looking for an experienced Data Engineer with strong expertise in Databricks, Azure, Scala, Python, and Apache Spark to design, develop, and maintain scalable data engineering solutions. The ideal candidate will have hands-on experience building data pipelines, processing large datasets, and working with cloud-based data platforms.

Key Responsibilities

• Design and develop scalable data pipelines and ETL/ELT workflows using Databricks and Apache Spark.
• Develop data engineering solutions using Python and Scala.
• Work with Azure cloud services for data ingestion, storage, processing, and integration.
• Build and optimize Spark-based data processing applications for large-scale datasets.
• Develop data transformation, cleansing, and validation processes.
• Collaborate with data architects, analysts, and application teams to deliver reliable data solutions.
• Troubleshoot performance, data quality, and pipeline issues.
• Follow best practices for data security, scalability, monitoring, and deployment.

Required Skills

• Strong hands-on experience with Databricks
• Strong knowledge of Microsoft Azure
• Expertise in Apache Spark
• Strong programming skills in Python and Scala
• Experience with ETL/ELT and data pipeline development
• Experience working with large-scale data processing and distributed computing
• Good understanding of cloud-based data engineering concepts
• Strong communication and problem-solving skills