Cloud/Bigdata

Imperio Inc.; Marietta (GA); On-site; USD 110,000 - 160,000; Full time

Job summary

Imperio Inc. is seeking a full-time, full-stack Data Engineer to design, develop, test, document and operate large-scale analytics data structures.

Responsibilities include real-time and batch ingestion using AWS tools, online reporting, and translating business needs into scalable data solutions. The role requires 4+ years in data warehousing, 7+ years in data warehouse architectures, and 3+ years Python/Java with Big Data stacks (EMR, Hadoop, MapReduce, Hive).

Qualifications

• 4+ years in a very large data warehousing environment.
• 7+ years of experience with data warehousing architectures, ETL/ELT and reporting/analytic tools.
• 3+ years of Python and/or Java development experience.
• 3+ years in Big Data stack environments (EMR, Hadoop, MapReduce, Hive).
• Excellent written and verbal communication skills.
• Ability to multi-task under pressure and work independently with minimal supervision.
• Strong ownership and collaboration within a team.

Responsibilities

• Design, develop, implement, test, document and operate large-scale data structures for analytics.
• Ingest data real-time and batch using AWS and Big Data tools.
• Provide online reporting and BI insights from multi-source datasets using BI tools.
• Translate business requirements into robust, scalable data solutions.
• Document datasets and metadata for future use.
• Advise on technical issues to project manager.
• Experience with Kafka, Flume, Redshift and Kinesis preferred.
• Write MapReduce and Spark jobs.
• Architect data warehouse solutions and integrate components.

Skills

Python

Java

SQL

Analytical skills

Communication skills

Multi-tasking

Ownership

Tools

Spark

Scala

Kinesis

Kafka

Redshift

Hadoop

MapReduce

Hive

Git

Jenkins

Subversion

Job description

\"The full stack Data Engineer will design, develop, implement, test, document, and operate large-scale, high-volume, high-performance data structures for analytics.\"

Primary Responsibilities:

• Well versed with BigData tools and technologies as Spark, Scala, Kinesis and Kafka.
• Implement data ingestion routines both real time and batch using best practices in data modelling, ETL/ELT processes leveraging AWS technologies and Big data tools.
• Provide on-line reporting and analysis using business intelligence tools and a logical abstraction layer against large, multi-dimensional datasets and multiple sources.
• Gather business and functional requirements and translate these requirements into robust, scalable, operable solutions that work well within the overall data architecture.
• Produce comprehensive, usable dataset documentation and metadata.
• Provides input and recommendations on technical issues to the project manager.
• Experience with Kafka, Flume and AWS tool stack such as Redshift and Kinesis are preferred.
• Experience using software version control tools (Git, Jenkins, Apache Subversion)
• AWS certifications or other related professional technical certifications
• Experience with cloud or on-premise middleware and other enterprise integration technologies
• Experience in writing MapReduce and/or Spark jobs
• Demonstrated strength in architecting data warehouse solutions and integrating technical components
• Good analytical skills with excellent knowledge of SQL.

Basic Qualifications:

• 4+ years of work experience with very large data warehousing environment
• Excellent communication skills, both written and verbal
• 7+ years of experience with detailed knowledge of data warehouse technical architectures, infrastructure components, ETL/ ELT and reporting/analytic tools.
• 3+ years of Python and/or Java development experience
• 3+ years' experience in Big Data stack environments (EMR, Hadoop, MapReduce, Hive)
• Flexible and proactive/self-motivated working style with strong personal ownership of problem resolution.
• Excellent communicator (written and verbal formal and informal).
• Ability to multi-task under pressure and work independently with minimal supervision.
• Strong verbal and written communication skills.

Must be a team player and enjoy working in a cooperative and collaborative team environment.