Systems Engineer - Data Pipelines & AWS (TS/Poly)

J5 Consulting; Virginia (MN); On-site; USD 100,000 - 120,000; Full time

Benefits offered by this job

100% employer-paid health coverage

6% 401(k) match

PTO

Tuition reimbursement

Bonuses

Professional development

Job summary

J5 Consulting is seeking a skilled data engineer to support operations for travel data applications. Candidates must have experience in building data pipelines, cloud computing, and tools like Apache Spark.

US Citizenship and an active U.S. Government Top Secret Security Clearance are mandatory. The role offers benefits including employer-paid health coverage, a 401(k) match, and professional development opportunities.

Qualifications

• Must have experience building and managing data pipelines.
• Proficient in Python programming.
• Experience with AWS services required.
• Knowledge of Apache Spark for data processing.
• Able to write SQL queries for RDBMS like Postgres or Oracle.
• Familiar with Linux and shell scripting.
• Experience analyzing various file formats like CSV and JSON.
• Must be able to write and validate unit tests.

Responsibilities

• Provide operations and maintenance for existing applications.
• Develop support for travel data applications.
• Support and develop travel data ETL pipelines.
• Assist with analytic capabilities of travel applications.

Skills

Data pipelining

Python

AWS services

Apache Spark

SQL queries

Linux and shell scripting

Data formats (CSV, XML, JSON, Avro, Parquet)

Unit tests

NiFi or Apache AirFlow

Java or Scala

EMR/Spark cluster administration

Performance tuning of Spark jobs

Supporting SQL access technologies

Cloud-based security solutions

Configuration management processes

Job description

J5 Consulting is seeking a skilled data engineer to support operations for travel data applications. Candidates must have experience in building data pipelines, cloud computing, and tools like Apache Spark.

US Citizenship and an active U.S. Government Top Secret Security Clearance are mandatory. The role offers benefits including employer-paid health coverage, a 401(k) match, and professional development opportunities.