Scala Data Engineer — Real-Time Streaming Pipelines

Compunnel Inc.; Greenwood Village (CO); On-site; USD 90,000 - 130,000; Full time

Job summary

A technology firm in Greenwood Village, Colorado is seeking a skilled Scala Developer to design and maintain scalable data pipelines for both real-time and batch processing. The ideal candidate will possess hands-on experience with Scala and familiarity with modern data engineering tools. This role involves contributing to large-scale data streaming projects and ensuring data quality throughout the ingestion and processing phases.

Qualifications

• Professional development experience with Scala.
• Practical knowledge of Apache Kafka for data streaming.
• Understanding of AWS cloud technologies (S3, Athena, EC2, IAM, EMR).

Responsibilities

• Develop and maintain data pipelines for real-time and hourly batch processing.
• Support scalable data flows for stakeholders, including reporting teams.
• Ensure data quality and integrity throughout ingestion and processing.

Skills

Scala programming

Apache Kafka

Spark Structured Streaming

AWS cloud technologies

Git

Job description

We are seeking a skilled Scala Developer to design and maintain scalable data pipelines for real-time and batch processing.

This role involves working with streaming technologies, cloud platforms, and big data frameworks to ensure efficient data ingestion, transformation, and delivery.

The ideal candidate will have hands-on experience with Scala and familiarity with modern data engineering tools.

Key Responsibilities

• Develop and maintain data pipelines for real-time and hourly batch processing.
• Support scalable data flows for internal and external stakeholders, including reporting teams and media partners.
• Implement new data exports and update business logic for various feeds.
• Contribute to the development of billion-scale streaming data pipeline infrastructure.
• Ensure data quality and integrity throughout ingestion and processing.

Required Qualifications

• Professional development experience with Scala.
• Practical knowledge of Apache Kafka for data streaming.
• Familiarity with Spark Structured Streaming (in Scala).
• Understanding of AWS cloud technologies (S3, Athena, EC2, IAM, EMR).
• Experience with Git for version control.

Preferred Qualifications

• Experience working on large-scale data streaming projects.
• Knowledge of additional big data frameworks and cloud security best practices.