Big Data Architect

GIGATEC Engineering; Columbia (MD); On-site; USD 110,000 - 170,000; Full time

Benefits offered by this job

100% Paid Healthcare

10% 401k in every paycheck

100% Fully Vested!

Job summary

GIGATEC Engineering seeks a Software Engineer to create and maintain data workflows and automation pipelines using Apache Airflow, delivering reliable and scalable solutions for data engineering and analytics.

You will collaborate with data engineers, platform teams, and stakeholders to ensure secure, production-ready workflows and observability across critical processes.

Qualifications

• Proficient in Linux CLI and Linux tools.
• Automation via Bash scripts for manual processes.
• Recent software development in Python and Java.
• Experience designing and operating Airflow DAGs and schedulers.
• Familiar with Spark for distributed data processing.
• Experience with Docker, containerd, Podman for containerization.
• Version control with Git.
• Familiarity with AWS cloud services and Jira/Confluence.

Responsibilities

• Create and maintain data workflows and automation pipelines using Apache Airflow.
• Collaborate with data engineers and platform teams to ensure reliability and observability.

Skills

Linux CLI

Bash scripting

Python/Java

Apache Airflow

Spark

Docker/Containerd/Podman

Git

AWS Cloud

JIRA/Confluence

Tools

Airflow tooling

Job description

Perks and Benefits

• 100% Paid Healthcare
• 10% 401k in every paycheck
• 100% Fully Vested!

NOTE – Our positions require a Top Secret clearance , as well as the favorable completion of a polygraph. Applicants must beauthorized to work in the U.S. We are unable to sponsor an employment Visa.

What You’ll Be Doing (We don’t love the bullet points, but we love the work!)

In this key Software Engineer role, you’ll be responsible for creating and maintaining data workflows and automation pipelines using Apache Airflow, keeping critical processes flowing as smoothly as a well-timed morning coffee. This role focuses on building reliable, scalable, and observable workflow orchestration solutions that support data engineering, analytics, and operational use cases, because no one wants a pipeline that is more decaf than dependable. You’ll get to collaborate with data engineers, platform teams, and stakeholders to ensure workflows are efficient, secure, and production ready, delivering results with the consistency of a favorite espresso order.

RequiredSkills

• Experience using the Linux CLI and Linux tools
• Experience developing Bash scripts to automate manual processes
• Recent software development experience using Python and Java
• Experience using Apache Airflow (DAG design, scheduling, operators, sensors) to orchestrate, schedule, and monitor complex workflows
• Experience using Distributed Big Data processing engines including Apache Spark
• Experience with containerization technologies such as Docker, containerd, and Podman
• Experience with Git Source Control System

RequiredSkills

• Familiar with AWS Cloud Services and Infrastructure
• Experience using the Atlassian Tool Suite (JIRA, Confluence)
• Fluent in sarcasm, kindness, and the occasional office banter