Data Engineer

RedBeard Solutions; Boston (MA); On-site; USD 120,000 - 170,000; Full time

Benefits offered by this job

Healthcare benefits

Job summary

RedBeard Solutions is seeking a Data Engineer to contribute to an AI-powered platform supporting data-driven solutions, including GenAI. You will partner with engineers, data scientists, and stakeholders to design scalable data infrastructure tailored for government and defense environments.

The role emphasizes cloud-native architectures, Python-based workflows, and robust data pipelines, with opportunities to work across AWS, Azure, and GCP in multi-cloud deployments.

Qualifications

• 5+ years of professional experience in data engineering with a focus on cloud-based solutions.
• Experience supporting defense or government environments, including handling sensitive data.

Responsibilities

• Design and build robust data ingestion and processing pipelines to support AI and GenAI applications.
• Develop and maintain Python-based data workflows, including testing and validation.
• Collaborate with cross-functional teams including software engineers, data scientists, and UI/UX designers.
• Implement and enhance CI/CD pipelines (GitHub Actions) to streamline deployment and integration.
• Work across multi-cloud environments (AWS primarily; exposure to Azure and GCP is beneficial).
• Utilize Docker and containerized workflows to support scalable deployments.
• Contribute to the development and optimization of data systems used by generative AI applications.
• Ensure performance, reliability, and scalability of data pipelines through monitoring and diagnostics.

Skills

Python

Cloud data engineering

Data pipelines

CI/CD

Docker

Knowledge graphs

Neo4j

Data modeling

Testing & validation

Communication

Tools

AWS

GitHub Actions

Docker

Azure

GCP

Job description

Location: Boston, Chicago, New York City, San Francisco, Silicon Valley, Washington DC

Number of openings: 1

Experience: 5+ years of professional experience as a data engineer

Position Summary

We are hiring a skilled Data Engineer to contribute to the development of a sophisticated AI-powered platform that supports advanced, data-driven solutions. This role offers the opportunity to work on cutting-edge initiatives involving generative AI, large-scale data systems, and cloud-native architectures.

You will partner with a multidisciplinary team of engineers, data scientists, and product stakeholders to design and implement scalable data infrastructure. The primary focus will be enabling platform capabilities tailored for government and defense-related environments.

Key Responsibilities

Design and build robust data ingestion and processing pipelines to support AI and GenAI applications

Develop and maintain high-quality Python-based data workflows, including testing and validation

Collaborate with cross-functional teams including software engineers, data scientists, and UI/UX designers

Implement and enhance CI/CD pipelines (including GitHub Actions) to streamline deployment and integration processes

Work across multi-cloud environments (AWS primarily; exposure to Azure and GCP is beneficial)

Utilize containerization technologies (Docker) to support scalable and portable deployments

Contribute to the development and optimization of data systems used by generative AI applications

Ensure performance, reliability, and scalability of data pipelines through monitoring and diagnostics

What You'll Experience

Opportunity to work on high-impact AI platform initiatives

A fast-paced, performance-driven environment with strong technical mentorship

Continuous learning through structured development programs and real-world problem solving

A collaborative culture that values innovation, ownership, and diverse perspectives

Exposure to globally distributed teams and modern engineering practices

Competitive compensation along with a comprehensive benefits package

Required Qualifications

U.S. Citizenship is required

Prior experience supporting defense or government environments, including handling classified or sensitive data

Active Secret clearance required (Top Secret or higher preferred)

5+ years of experience in data engineering, with a strong focus on cloud-based solutions

Technical Skills & Expertise

Strong proficiency in Python for data engineering and pipeline development

Hands-on experience with AWS-based data engineering (experience with Azure or GCP is a plus)

Deep understanding of data modeling, data ingestion frameworks, and pipeline architecture

Experience working with knowledge graphs, particularly using Neo4j, and familiarity with relational databases

Expertise in writing clean, maintainable, and testable code, including automation and error handling best practices

Practical experience with Docker and containerized workflows

Strong background in data pipeline performance tuning, monitoring, and troubleshooting

Excellent analytical and problem-solving capabilities

Effective communication skills and ability to collaborate within cross-functional teams