A technology firm in Dearborn, Michigan, is seeking a skilled developer to design and develop data ingestion pipelines. The ideal candidate will have expertise in Java and Hadoop, with a strong focus on automating ETL processes. Collaborative work with Business Analysts and Senior Developers is essential to establish the application framework. Candidates should be proficient in various web application technologies and data management tools. This role promises a dynamic environment with opportunities to drive innovative solutions.
Responsibilities
Design and develop data ingestion pipelines.
Perform data migration and conversion activities.
Develop and integrate software applications.
Collaborate with stakeholders to establish the application framework.
Automate ETL processes for various datasets.
Skills
Java
J2EE
Web Applications
Tomcat
Restful Services
JSON
Hadoop
HDFS
Hive
Impala
Spark
Oozie
HBase
Job description
Essential Job
Design and development of data ingestion pipelines.
Perform data migration and conversion activities.
Develop and integrate software applications using suitable development methodologies and standards, applying standard architectural patterns, taking into account critical performance characteristics and security measures.
Collaborate with Business Analysts, Architects and Senior Developers to establish the physical application framework (e.g. libraries, modules, execution environments).
Perform end to end automation of ETL process for various datasets that are being ingested into the big data platform.
Requirements
Java
J2EE, Web Applications, Tomcat (or any equivalent App server), Restful Services, JSON