Senior Data Engineer

Apexon; Great Falls Crossing (VA); Hybrid; USD 100,000 - 130,000; Full time

Job summary

A leading IT consulting firm is seeking a Mid-Senior Level Data Engineer with extensive experience in Java, Python, and Big Data technologies. The role primarily involves integrating data using Hadoop and AWS services, with programming skills in Kafka being crucial. This position allows for in-person work once a week in Reston, Virginia, and requires a commitment to code quality and automated testing.

Qualifications

• 7+ years of programming experience with Java/Python/Scala.
• 3+ years working on Data Integration projects using Hadoop.
• 2+ years of experience on AWS services like Lambda and S3.
• Experience with Kafka-based data ingestion programs.

Responsibilities

• Tune Hadoop/Spark/Hive for optimal performance.
• Write SQL queries and perform data analysis.
• Implement engineering best practices in code quality.

Skills

Java programming

Python

Scala

Hadoop

AWS services

Kafka

SQL

Shell scripting

Job description

Need consultants from Virginia , DC , Maryland, and West Virginia

Job Description :

Requirements

• 7+ years of strong programming background with Java/Python/Scala
• At least 3+ years of experience working on Data Integration projects using Hadoop MapReduce, Sqoop, Oozie , Hive, Spark and other related Big Data technologies
• At least 2+ years of experience on AWS preferably leveraging services such as Lambda, S3, Redshift, Glue services
• Some working experience building Kafka based data ingestion/retrieval programs
• Experience tuning Hadoop/Spark/hive parameters for optimal performance
• Strong SQL query writing and data analysis skill
• Good shell scripting experience
• Rigor in high code quality, automated testing, and other engineering best practices, ability to write reusable code components
• Big data engineer role with solid AWS experience; once a week in person at Reston office mostly.

Notes

• This position requires strong Java programming skills along with other skills already listed on the req.
• Data engineers with a good background in Hadoop and solid AWS as mostly of the work is being migrated to AWS.
• Kafka is IMP and they should know of the programming languages like Java

Seniority level

• Mid-Senior level

Employment type

• Full-time

Job function

• Information Technology

Industries

• IT Services and IT Consulting

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.