Data Engineer for Big Data & AI Pipelines (TS/SCI)

AVIAN; Virginia (MN); On-site; USD 110,000 - 170,000; Full time

Benefits offered by this job

Health insurance

Dental & vision

Telemedicine

401(k) with company contributions

ESOP

Wellness reimbursements

Referral bonuses

Life insurance

Paid leave

Tuition reimbursement

Professional development

Pet insurance

Identity protection

Life assistance programs

Job summary

Level Up, LLC, a subsidiary of AVIAN, seeks a data engineering professional to design, ingest, cleanse, transform, and display large data sets in a Big Data environment. You will work with Python/Java, SQL, and data pipelines to support AI/ML workflows.

You will collaborate with data scientists, developers, and vendors to build scalable data solutions while adhering to security and compliance requirements, including TS/SCI with Full Scope Polygraph.

Qualifications

• High School Diploma/GED with 2+ years of relevant software development/programming experience
• Demonstrated data analysis, parsing, and programming language experience (e.g. Python, Java) coupled with significant SQL/database experience
• Experience with the full data lifecycle, from ingest through display, in a Big Data environment
• Hands-on experience with Java-related technologies, such as JDK, J2EE, EJB, JDBC, and/or Spring, and experience with RESTful APIs
• Experience with data pipelining systems (e.g. Apache Airflow) and developing/performing ETL tasks in a Linux environment

Responsibilities

• Designing, implementing, and optimizing data extraction, cleansing, transformation, loading, replication/distribution, and large-scale ingest systems in a Big Data environment
• Optimizing all stages of the data lifecycle, from initial planning, to ingest, through final display and beyond
• Developing custom solutions/code to ingest and exploit new and existing data sources
• Developing data profiling, deduping logic, and matching logic for analysis
• Organizing and maintaining Data Layer documentation, so others are able to understand and use it. Also, work closely with data scientists to craft data pipelines which serve the development of modern AI/ML workflows
• Collaborating with teammates, other service providers, vendors, and users to develop new and more efficient methods
• Effectively articulating the risks and constraints associated with software solutions, based on environment

Skills

Python

Java

SQL

Data analysis

Big Data

AI/ML workflows

Data lifecycles

Education

High School Diploma/GED

Tools

Apache Airflow

Linux

RESTful APIs

JDK/J2EE

Job description

Level Up, LLC, a subsidiary of AVIAN, seeks a data engineering professional to design, ingest, cleanse, transform, and display large data sets in a Big Data environment. You will work with Python/Java, SQL, and data pipelines to support AI/ML workflows.

You will collaborate with data scientists, developers, and vendors to build scalable data solutions while adhering to security and compliance requirements, including TS/SCI with Full Scope Polygraph.