DS Data Engineer

Probity Inc.; Chantilly (VA); On-site; USD 80,000 - 120,000; Full time

Job summary

Probity Inc. is seeking a Data Engineer in Chantilly, VA, requiring an ACTIVE Top Secret/SCI Clearance. The role involves designing and optimizing data systems, implementing data lifecycle processes, and collaborating with teams to enhance methods.

Successful candidates will have at least 2 years of software development experience, proficiency in programming languages like Python and Java, and expertise with SQL in a Big Data environment.

Qualifications

• 2+ years of relevant software development/programming experience.
• Strong data analysis and programming language skills required.
• Experience with the full data lifecycle in a Big Data environment.

Responsibilities

• Design and optimize data extraction and transformation systems.
• Develop solutions to ingest and exploit new data sources.
• Collaborate to improve methods and articulate risks associated with software solutions.

Skills

Data analysis

Python

Java

SQL

ETL

Apache Airflow

Education

High School Diploma/GED

Tools

Java-related technologies

RESTful APIs

Job description

PLEASE NOTE: This position requires an ACTIVE Top Secret/SCI Clearance with Polygraph. To be considered for this position, you MUST have an ACTIVE Clearance Level of Top Secret/SCI with Polygraph

Position Code: 04-DM0522-1

Location: Chantilly, VA

Summary

The Data Engineer will leverage their development skills and experience, to support the successful designing, ingesting, cleansing, transformation, loading, and display of significant amounts of data.

Duties, Tasks & Responsibilities

• Designing, implementing, and optimizing data extraction, cleansing, transformation, loading, replication/distribution, and large-scale ingest systems in a Big Data environment
• Optimizing all stages of the data lifecycle, from initial planning, to ingest, through final display and beyond
• Developing custom solutions/code to ingest and exploit new and existing data sources
• Developing data profiling, deduping logic, and matching logic for analysis
• Organizing and maintaining Data Layer documentation, so others are able to understand and use it. Also, work closely with data scientists to craft data pipelines which serve the development of modern AI/ML workflows
• Collaborating with teammates, other service providers, vendors, and users to develop new and more efficient methods
• Effectively articulating the risks and constraints associated with software solutions, based on environment

Required Experience, Skills, & Technologies

• High School Diploma/GED with 2+ years of relevant software development/programming experience.
• Demonstrated data analysis, parsing, and programming language experience (e.g. Python, Java) coupled with significant SQL/database experience.
• Experience with the full data lifecycle, from ingest through display, in a Big Data environment.
• Hands‑on experience with Java‑related technologies, such as JDK, J2EE, EJB, JDBC, and/or Spring, and experience with RESTful APIs.
• Experience with data pipelining systems (e.g. Apache Airflow) and developing/performing ETL tasks in a Linux environment.

Desired Experience, Skills & Technologies

• Experience deploying systems that leverage AI/ML technology
• Experience publishing results in BI dashboards.