Data Engineer: Big Data Pipelines & AI-Ready Analytics

AVIAN; Virginia (IL); On-site; USD 90,000 - 130,000; Full time

Benefits offered by this job

Medical, dental, vision

401(k) with company contributions

ESOP

Wellness reimbursements

Referral bonuses

Life insurance

Disability benefits

AFLAC supplemental insurance

Flexible time off

Paid sick leave

Maternity leave

Tuition reimbursement

Professional development

Pet insurance

Identity protection

Life assistance

Job summary

Level Up, LLC, a wholly-owned subsidiary of AVIAN, seeks a data-focused software professional to design and optimize big data ingestion, cleansing, and transformation pipelines. You will work on end-to-end data workflows from source to display, collaborating with data scientists and cross-functional teams.

Candidates should have 2+ years in software development with Python/Java, SQL proficiency, and experience with Airflow and ETL processes in Linux environments.

Qualifications

• Must have at least a high school diploma or GED with 2+ years of software development experience.
• Experience with data analysis, parsing, and programming languages (Python, Java) and strong SQL/database experience.
• Experience with the full data lifecycle from ingest to display in a Big Data environment.
• Hands-on experience with Java technologies (JDK/J2EE/EJB/JDBC/Spring) and RESTful APIs.
• Experience with data pipelining systems (e.g., Apache Airflow) and ETL tasks in a Linux environment.

Responsibilities

• Design, implement, and optimize data extraction, cleansing, transformation, loading, replication/distribution, and large-scale ingest systems in a Big Data environment.
• Optimize all stages of the data lifecycle from ingestion to display.
• Develop custom solutions to ingest and leverage new data sources.
• Develop data profiling, deduping, and matching logic for analysis.
• Maintain Data Layer documentation and collaborate with data scientists to craft data pipelines for AI/ML workflows.
• Collaborate with teammates, service providers, vendors, and users to develop efficient methods.
• Articulate risks and constraints of software solutions based on environment.

Skills

Data analysis

SQL/Database

Python

Java

REST APIs

Airflow

ETL

Big Data

Education

High School Diploma or GED

Tools

JDK

J2EE

EJB

JDBC

Spring

Job description

Level Up, LLC, a wholly-owned subsidiary of AVIAN, seeks a data-focused software professional to design and optimize big data ingestion, cleansing, and transformation pipelines. You will work on end-to-end data workflows from source to display, collaborating with data scientists and cross-functional teams.

Candidates should have 2+ years in software development with Python/Java, SQL proficiency, and experience with Airflow and ETL processes in Linux environments.