Big Data Engineer (W2 Candidates only)

360 IT Professionals; Atlanta (GA); On-site; USD 130,000 - 160,000; Full time

Job summary

A leading software development firm in Atlanta is looking for a Big Data Engineering Lead to manage the ingestion and governance of data. The role requires a strong background in distributed computing and experience with the Hadoop Ecosystem. Candidates must have a passion for software engineering and proficiency in Java and Scala. The position offers an opportunity to work in a collaborative environment focused on driving technical excellence and innovation.

Qualifications

• Bachelor’s degree or higher in Computer Science or related field.
• Proven experience in developing Big Data solutions.
• Good understanding of distributed computing and big data architectures.

Responsibilities

• Lead a Big Data engineering team focusing on data ingestion.
• Participate in the design and development of the new data lake.
• Inspire the team and cultivate engineering mindset.

Skills

Proficient in distributed computing

Experience with Apache NiFi

Knowledge of NoSQL databases

Understanding of Agile methodologies

Capable of developing in Java

Strong coding skills

Education

Bachelor’s degree in Computer Science

Tools

Hadoop Ecosystem

ETL tools (Talend, Pentaho)

Docker

Git

Job description

360 IT Professionals is a Software Development Company based in Fremont, California that offers complete technology services in Mobile development, Web development, Cloud computing and IT staffing. Merging Information Technology skills in all its services and operations, the company caters to its globally positioned clients by providing dynamic feasible IT solutions. 360 IT Professionals work along with its clients to deliver high-performance results, based exclusively on the one of a kind requirement.

Our services are vast and we produce software and web products. We specialize in Mobile development, i.e. iPhone and Android apps. We use Objective-C and Swift programming languages to create native applications for iPhone, whereas we use Android Code to develop native applications for Android devices. To create applications that work on cross-platforms, we use a number of frameworks such as Titanium, PhoneGap and Jquery mobile.

Furthermore, we build web products and offer services such as web designing, layouts, responsive designing, graphic designing, web application development using frameworks based on model view controller architecture and content management system. Our services also extend to the domain of Cloud Computing, where we provide Salesforce CRM to effectively manage one’s business and ease out all the operations by giving an easy platform. Apart from this, we also provide IT Staffing services that can help your organization to a great extent as you can hire highly skilled personnel’s through us.

We make sure that we deliver performance-driven products that are optimally developed as per your organization’s needs. Take a shot at us for your IT requirements and experience a radical change.

Job Description

Responsibilities

• Lead Big Data engineering team with specialization on data ingestion (from 100+ source systems in batch and near real-time), egestion and governance.
• Participate in collaborative software and system design and development of the new NCR Data Lake on Hortonworks HDP and HDF distributions.
• Manage own learning and contribute to technical skill building of the team.
• Inspire and cultivate the engineering mindset and systems thinking.
• Gain deep technical expertise in the data movement patterns, practices and tools.
• Play active role in Big Data Communities of Practice.
• Put the minimal system needed into production.

Required Qualifications

• Bachelor’s degree or higher in Computer Science or a related field.
• Good understanding of distributed computing and big data architectures.
• Passion for software engineering and craftsman‑like coding prowess.
• Proven experience in developing Big Data solutions in Hadoop Ecosystem using Apache NiFi, Kafka, Flume, Sqoop, Apache Atlas, Hive, HDFS, HBase and Spark (Hortonworks HDP and HDF preferred).
• Experience with at least one of the leading CDC (Change Data Capture) tools like Informatica PowerCenter.
• Development experience with at least one NoSQL database. HBase or Cassandra preferred.
• Polyglot development (4-5 years+): Capable of developing in Java and Scala with good understanding of functional programming, SOLID principles and, concurrency models and modularization.
• DevOps: Appreciates the CI and CD model and always builds to ease consumption and monitoring of the system. Experience with Maven (or Gradle or SBT) and Git preferred.
• Experience in Agile development including Scrum and other lean techniques.
• Should believe in You Build! You Ship! And You Run! Philosophy.
• Personal qualities such as creativity, tenacity, curiosity, and passion for deep technical excellence.

Desired Qualifications

• Experience with Big Data migrations/transformations programs in the Data Warehousing and/or Business Intelligence areas.
• Experience with ETL tools like Talend, Pentaho, Attunity etc.
• Knowledge of Teradata, Netezza etc.
• Good grounding in NoSQL data stores such as Cassandra, Neo4j etc.
• Strong knowledge on computer algorithms.
• Experience with workload orchestration and automation tools like Oozie, Control-M etc.
• Experience in building self-contained applications using Docker, Vagrant. Chef.