Data Engineer

Allnessjobs; Washington (District of Columbia); On-site; USD 110,000 - 115,000; Full time

Benefits offered by this job

Medical Insurance

Dental Insurance

Profit-Sharing Program

Job summary

A leading consulting firm in the Washington, DC area is seeking a Data Engineer to design and develop data ingestion and cleaning processes using Python and Spark. The role requires 5+ years of big data experience, strong knowledge in data modeling, and the ability to communicate technical issues to non-technical staff. The position includes competitive pay and benefits like medical and dental insurance.

Qualifications

• 5+ years of experience with big data.
• 3+ years of experience with software development lifecycle.
• 5+ years of Python coding experience.

Responsibilities

• Design and develop data ingestion and cleaning processes.
• Create big data pipelines for structured and unstructured data.
• Support data migration activities and improve database performance.

Skills

Python

Big Data

Data Modeling

UNIX

CI/CD Automation

Education

Bachelor's degree

Tools

Hadoop

Spark

AWS

Databricks

Job description

Overview

Join to apply for the Data Engineer role at Allnessjobs.

This range is provided by Allnessjobs. Your actual pay will be based on your skills and experience — talk with your recruiter to learn more.

We have partnered with a large consulting firm in the Washington, DC area to provide them with a Data Engineer.

Base pay range

$110,000.00/yr - $115,000.00/yr

Responsibilities

• Designing and developing ongoing data ingestion and cleaning using Python and Spark.
• Support Requirements gathering activities, especially from the data migration perspective.
• Using programming skills to create big data pipelines that can deal with both structured and unstructured data.
• Support other teams' ineffective use of big data tools and provide guidance in writing efficient code in Python and/or R.
• Analyzes and loads data for use by a larger audience.
• Analyzes databases to improve the speed of data access.
• Writes views for consultants to simplify data access.
• Diagnoses and corrects database performance bottlenecks.
• Analyzes data provided by project consultants and makes it available in our database; provides access to the database to those that have a need.
• Develops strategies for data acquisitions, archive, recovery, and implementation of a database.
• Maintains the database and cleans data as required.
• Designs and develops databases, data warehouses, and multidimensional databases.
• Leads and directs data management work of others as applicable.
• Reports to the office on a regular basis to allow for in-person interactions including providing oversight and mentorship to the team, attending meetings with other employees, candidates, and vendors, participating in performance conversations, attending firm meetings, or as otherwise requested by the direct supervisor.

Requirements

• Bachelor's degree required.
• 5+ years of experience working with big data.
• 3+ years of experience with software development lifecycle.
• 3+ years of developing data ingestion and transformation processes associated with big data technologies (e.g. Hadoop, Spark, etc.)
• 5+ years of hands-on Python coding experience.
• Experience with data modeling, integration, and warehousing.
• Hands-on experience with CI/CD automation is preferred.
• Working knowledge with UNIX variants preferred.
• Previous experience working with Cloud (e.g. AWS) is a plus.
• Experience with Databricks is a plus
• Experience with performance tuning of ETL jobs.
• Ability to communicate technical issues to non-technical staff.
• Ability to perform essential functions with or without reasonable accommodation.
• May require more than 40.0 hours per week to perform the essential duties of the position.

Benefits

• Medical Insurance
• Dental Insurance
• Profit-Sharing Program

Seniority level

• Mid-Senior level

Employment type

• Full-time

Job function

• Information Technology

Industries

• Human Resources Services