Senior Big Data Engineer

Deegit Inc; Salt Lake City (UT); On-site; USD 100,000 - 130,000; Full time

Job summary

A technology company in Salt Lake City, Utah, is seeking an experienced professional to engineer and administer big data systems. The role requires extensive experience with Java programming, data transformation pipelines, and knowledge of operational analytics. You will collaborate with teams to optimize data collection and maintain high availability of big data technologies. Ideal candidates will have a BS in Computer Science and experience with tools like Apache Storm and Splunk.

Qualifications

• 5+ years’ experience with data transformation pipelines, preferably Apache Storm.
• 5+ years’ experience in messaging frameworks, preferably Kafka.
• 5+ years’ experience with Java programming.

Responsibilities

• Participate in the engineering and administration of big data systems.
• Collaborate with teams to optimize data collection.
• Proactively monitor performance.

Skills

Big data systems

Apache Storm

Java programming

SQL

Operational analytics

Messaging frameworks

Deployment automation

Education

BS in Computer Science

Tools

ELK

Splunk

Greenplum

Hadoop technologies

Job description

Responsibilities

• Participate in the engineering and administration of big data systems.
• Apache Storm/Java development for both data transformation and augmentation.
• Employ best practices to ensure high availability and scalability.
• Install, configure, and maintain big data technologies and systems.
• Maintain documentation and troubleshooting playbooks.
• Respond to and resolve access and performance issues.
• Ability to work non-peak hours when needed.
• Assist in capacity planning.
• Proactively monitor performance.
• Collaborate with engineering teams to optimize data collection.
• Maintain change control and testing processes for all modifications and deployments.
• Conduct research and make recommendations on big data products, services, and standards.

Preferred Qualifications

• Experience with operational analytics and log management using ELK or Splunk.
• Understanding and experience with SQL and MPP databases such as Greenplum.
• Experience with SaltStack or comparable deployment automation tools.
• Working knowledge of Zenoss, NewRelic, PagerDuty or other monitoring technologies.
• Experience with Apache Drill, Apache HBase, Hive, Phoenix, or other Hadoop with SQL technologies.
• Experience working with batch-processing and tools in Hadoop technology stack (MapReduce, Yarm, Pig, Hive, HDFS).

Qualifications

Basic Requirements

• BS in Computer Science, Information Systems, or related technical degree.
• 5 or more years’ experience with data transformation pipelines preferably Storm.
• 5 or more years’ experience in messaging frameworks preferably Kafka.
• 5 or more years’ experience with Java programming.

Additional Information

All your information will be kept confidential according to EEO guidelines.