Senior Core Data Engineer

SA TECHNOLOGIES; San Francisco (CA); On-site; USD 120,000 - 160,000; Full time

Job summary

A leading IT consulting firm in San Francisco is seeking a Senior Core Data Engineer with strong Java programming skills and experience in Hadoop MapReduce and NoSQL databases like HBase or Cassandra. The role involves ingesting large datasets, processing them, and serving the outcomes to applications. Ideal candidates will have a hands-on background in data engineering and relevant technologies. This position offers the chance to work on exciting projects in a dynamic environment.

Qualifications

• Solid Java programmer with production-level experience on Hadoop MapReduce.
• Hands-on experience with HBase or Cassandra databases.
• Ability to ingest large amounts of raw data and run processing jobs.

Responsibilities

• Ingest large amounts of raw data like web logs and data feeds.
• Run Hadoop MapReduce jobs to parse and process data.
• Load outcomes to NoSQL databases and serve to applications.

Skills

Java Programming

Hadoop MapReduce

HBase

Cassandra

Spring Boot

Job description

SA Technologies Inc. (www.satincorp.com) is a market leader and one of the fastest growing IT consulting firms with operations in US, Canada, Mexico & India. SAT is an Oracle Gold Partner, SAP Services Partner & IBM Certified enterprise.

All SA Technologies requirements are Direct Client Requirements from IT Hiring Managers. We guarantee you the best rate for your skills and performance.

Job Description

Position: Senior Core Data Engineer

Location: SFO, CA

Duration: 12 Months

Solid Java programmer who has production-level hands‑on experience on Hadoop MapReduce and HBase or Cassandra database.

Examples of previous project experience we are looking for:

• Ingested large amount of raw data (web logs, web pages, data feeds, …)
• Run Hadoop MapReduce jobs to parse them and process them (simple reporting, machine learning, …)
• Loaded the outcome to NoSQL database (HBase / Cassandra) and served them to other applications (live website traffic, other internal tools, …)

Technologies that are a must:

• Programming language -- Java
• Batch processing -- Hadoop MapReduce
• Restful web services -- Spring Boot, Jackson

Technologies that would get our attention…

• Programming language:
  • Python
  • Scala
  • Shell scripting
• Batch Processing:
  • Cascading/Scalding
  • Apache Spark
• Stream processing -- AKKA, Samza, Spark streaming
• NoSQL -- MongoDB, Riak, …
• Code/Build/Deployment -- hg, svn, sbt, bamboo, …

Additional Information

All your information will be kept confidential according to EEO guidelines.