Senior Data Engineer - Scala/Spark, Medicaid Data Platform

TALENT Software Services; Lincoln (NE); On-site; USD 120,000 - 160,000; Full time

Job summary

TALENT Software Services in Lincoln, NE seeks a Senior Data Engineer to be the primary technical owner for the enterprise data processing framework supporting Medicaid Encounter Processing and data ingestion.

The role focuses on maintaining the production code base, implementing regulatory changes, supporting operations, resolving defects, and ensuring reliability across Scala, Spark, Hive, Drools, and Cloudera Data Platform.

Qualifications

• 7+ years of experience developing enterprise-scale distributed data processing applications.
• Strong hands-on development experience with Scala.
• Experience developing applications using Apache Spark 2.x and/or Spark 3.x.
• Experience developing and maintaining applications running on Cloudera Data Platform (CDP) 7.x or equivalent enterprise Hadoop distributions.
• Experience with Apache Hive 3.x for data processing and Hive table management.
• Experience implementing business rules using the Drools Rules Engine.
• Strong SQL development and query optimization skills.
• Experience supporting Linux-based production environments.
• Experience troubleshooting distributed Spark applications in production.
• Experience using Git and modern version control practices.

Responsibilities

• Own the Scala/Spark application framework for Medicaid Encounter Processing.
• Own the Drools rule implementation and ongoing rule maintenance.
• Oversee enterprise data ingestion processes and batch processing with Spark and Hive.
• Lead production issue investigation and defect resolution.
• Coordinate production releases, version management, and deployments.
• Provide performance tuning and optimization across the stack.
• Maintain technical documentation and support knowledge transfer.
• Provide basic operational administration and monitoring of the Cloudera Data Platform.

Skills

Scala

Apache Spark

Hive

Drools

SQL

Linux

Git

Production support

Performance tuning

Tools

Cloudera Data Platform (CDP)

HDFS

YARN

IBM DataStage

AWS

Job description

TALENT Software Services in Lincoln, NE seeks a Senior Data Engineer to be the primary technical owner for the enterprise data processing framework supporting Medicaid Encounter Processing and data ingestion.

The role focuses on maintaining the production code base, implementing regulatory changes, supporting operations, resolving defects, and ensuring reliability across Scala, Spark, Hive, Drools, and Cloudera Data Platform.