Senior Data Engineer - Encounter Processing & Data Platform

Zohorecruit; Oklahoma; On-site; USD 120,000 - 180,000; Full time

Job summary

Zohorecruit in Lincoln, United States, seeks a Senior Data Engineer to own the maintenance, enhancement, and reliability of an enterprise data processing framework supporting Medicaid Encounter Processing.

You will work with Scala, Spark, Hive, and Drools on the Cloudera Data Platform, ensuring performance, scalability, and production stability across healthcare data pipelines.

Qualifications

• 7+ years of experience developing enterprise-scale distributed data processing applications.
• Senior Data Engineer responsible for maintenance, enhancement, operational support, and reliability of data processing framework for Medicaid Encounter Processing.
• Platform processes provider, member, reference, and encounter data using Scala, Spark, Hive, Drools on CDP.

Responsibilities

• Own, maintain, and enhance the Scala/Spark application framework.
• Develop, maintain, and enhance Drools business rules.
• Manage data ingestion processes for master data and source files.
• Develop and support Spark and Hive batch processing workflows.
• Troubleshoot production issues and resolve defects.
• Perform operational administration and monitoring of CDP.

Skills

Scala

Apache Spark

Apache Hive

Drools

SQL

Cloudera Data Platform

HDFS/YARN

AWS

Tools

IBM DataStage

Cloudera Data Platform

Apache Hadoop

Cloudera Manager

AWS

Job description

Lincoln, United States

• Help design, build and continuously improve the clients online platform.
  
• Research, suggest and implement new technology solutions following best practices/standards.
  
• Take responsibility for the resiliency and availability of different products.
  
• Be a productive member of the team.
  

Requirements

• 7+ years of experience developing enterprise-scale distributed data processing applications.
  
• Senior Data Engineer to serve as the primary technical owner for the maintenance, enhancement, operational support, and reliability of an enterprise data processing framework supporting Medicaid Encounter Processing and enterprise data ingestion.
  
• The platform processes provider, member, reference, and encounter data for downstream Medicaid systems and reporting using Scala, Apache Spark, Hive, and Drools on the Cloudera Data Platform (CDP).
  
• Own, maintain, and enhance the Scala/Spark application framework supporting Medicaid Encounter Processing.
  
• Develop, maintain, and enhance Drools business rules based on evolving business requirements.
  
• Manage enterprise data ingestion processes for master data and source-system files.
  
• Develop, maintain, and support Spark and Hive batch processing workflows.
  
• Investigate production issues, troubleshoot application defects, and implement effective resolutions.
  
• Perform operational administration, monitoring, and troubleshooting of the Cloudera Data Platform.
  
• Ensure data processing reliability, performance, scalability, and operational stability.
  
• Support continuous improvement of data processing frameworks and production support processes.
  
• Strong hands-on experience with Scala.
  
• Strong experience with Apache Spark 2.x and/or 3.x.
  
• Experience with Apache Hive 3.x, including data processing and Hive table management.
  
• Hands-on experience implementing business rules using the Drools Rules Engine.
  
• Strong SQL development and query optimization skills.
  
• Experience supporting and troubleshooting enterprise data processing applications in production environments.
  
• Experience with Medicaid or healthcare data, particularly Medicaid Encounter Processing.
  
• Experience with Cloudera Manager, HDFS, and YARN.
  
• Experience integrating with IBM DataStage.
  
• Familiarity with AWS infrastructure supporting Cloudera.
  
• Experience supporting enterprise-scale data platforms in production environments.
  
• A challenging, innovating environment.
  
• Opportunities for learning where needed.