Senior Data Engineer: Real-Time Pipelines & Spark

InfoVision Inc.; Denver (CO); On-site; USD 110,000 - 126,000; Full time

Job summary

A technology consulting firm in Denver is seeking a Data Engineer with data engineering experience to build ETL pipelines and manage data flows using Spark, Kafka, and various data formats including JSON and Parquet. This role requires strong analytical skills and proficiency in cloud technologies like AWS. The candidate should have experience in managing data pipelines for analytics. This position offers a competitive salary range of approximately $110,000 to $126,000 with contract employment.

Responsibilities

• Build ETLs using Spark and SQL.
• Manage real-time and batch data pipelines using Kafka.
• Work with multiple data formats such as JSON, Parquet, and Delta files.

Skills

Strong organizational and analytical skills

Experience implementing and maintaining high-performance data structures

Experience in relational SQL and NoSQL databases

High proficiency in Python, Spark, SQL, and/or Scala

Experience in designing and implementing ETL processes

Experience in managing data pipelines

Strong understanding of in-memory processing

Experience in AWS cloud services

Experience in stream-processing systems

Experience in data pipeline management tools

Experience in big data tools

Experience in software engineering tools and practices

Experience in data observability and monitoring

Hands-on experience in maintaining data-schema life-cycles

Good to have experience with Databricks and Snowflake

Job description

A technology consulting firm in Denver is seeking a Data Engineer with data engineering experience to build ETL pipelines and manage data flows using Spark, Kafka, and various data formats including JSON and Parquet. This role requires strong analytical skills and proficiency in cloud technologies like AWS. The candidate should have experience in managing data pipelines for analytics. This position offers a competitive salary range of approximately $110,000 to $126,000 with contract employment.