Principal Software Engineer - Workload Communications Domain Lead

SciTec; Princeton (NJ); On-site; USD 164,000 - 218,000; Full time

Benefits offered by this job

401k match

Health insurance

Vision insurance

Life insurance

Disability insurance

Profit sharing

Parental leave

Paid time off

Flexible work hours

Job summary

SciTec, a Firefly Aerospace company, seeks a Principal Software Engineer to lead workload communications domain, guiding design of data and control flow across processes and services. The role blends hands-on coding with technical stewardship to improve distributed workloads for mission-critical data processing and visualization.

The ideal candidate will have deep experience with messaging systems (Kafka, RabbitMQ, ActiveMQ, MQTT) and RPCs (gRPC/ZeroMQ), strong delivery semantics, and the

Qualifications

• Bachelor’s degree in Computer Science, Engineering, Mathematics, Physical Sciences, or a related technical field.
• 10+ years of professional experience with distributed communication systems.
• Experience with one or more message brokers or event streaming platforms (e.g., Kafka, RabbitMQ, ActiveMQ, MQTT).
• Experience with one or more RPC or low-level messaging frameworks (e.g., gRPC, ZeroMQ).
• Strong understanding of delivery semantics, ordering, idempotency, backpressure, and fault handling in distributed systems.
• Experience with serialization formats and schema evolution (e.g., Protocol Buffers, Avro, FlatBuffers, JSON Schema).
• Proficiency in at least one backend language such as C++, Python, Go, Rust, Java, or TypeScript/Node.js.
• Comfortable working in Linux development environments.
• Strong technical writing, mentoring, and cross-team communication skills with ability to influence engineering practices through technical credibility.
• Ability to obtain and maintain a DoD security clearance.

Responsibilities

• Assess current communication technologies, patterns, and gaps across teams.
• Define evaluation criteria and run load tests to select core communication technologies and patterns.
• Document recommended technologies and ADRs with guides and references.
• Build validated reference implementations and starter templates for common use cases.
• Guide choice between synchronous, asynchronous, event streaming, and other patterns.
• Establish practices for delivery guarantees, ordering, retries, deduplication, and idempotency.
• Guide backpressure, buffering, batching, flow control, and concurrency design.
• Optimize throughput, latency, serialization overhead, network utilization, and memory pressure.
• Establish schema evolution practices for data contracts.
• Guide partitioning, fan-out, replay, retention, and consumer-group design.
• Establish fault handling, dead-letter processing, observability, and recovery practices.
• Collaborate with engineers through pair programming and reviews to drive adoption of best practices.
• Work with Data Storage and Deployment domains on cross-domain issues like consumer lag and tracing.
• Periodically reassess and update recommendations as tech and workloads change.
• Track adoption and outcomes against baselines and report to leadership.

Skills

Distributed messaging

Event streaming

RPC systems

Backpressure and fault handling

Linux development

Code review and mentoring

Education

Bachelor’s degree in CS/Engineering/Math/Physics

Tools

Kafka

RabbitMQ

ActiveMQ

MQTT

gRPC

ZeroMQ

Job description

SciTec, a Firefly Aerospace company is a dynamic non-traditional defense contractor that delivers advanced technologies in support of U.S. National Security and Defense. For the past forty-five plus years, we have supported Department of Defense customers by developing innovative remote sensing algorithms, tools, and techniques to deliver world-class data exploitation capabilities supporting missile defense; intelligence, surveillance, & reconnaissance; space domain awareness; and aircraft survivability missions.

Important Notice: SciTec exclusively works on U.S. government contracts that require U.S. citizenship for all employees. Applicants that do not meet this requirement will not be considered.

We are seeking a Principal Software Engineer to serve as our Workload Communications Domain Lead. In this role, you will guide how our engineering teams design and optimize the flow of data and control between processes, services, containers, and other distributed workloads that support mission critical data processing, visualization, and operational workflows. This role is hands on and combines deep technical work with technical stewardship: you will assess how our teams communicate between workloads today, establish evidence-based recommendations, build reference implementations, and work directly with engineers to raise our organization-wide capability.

The ideal candidate has deep, production-grade expertise in distributed messaging, event streaming, and RPC systems, the ability to select communication patterns based on real workload requirements, and the communication and mentoring skills to turn that expertise into practical guidance that other engineers can adopt independently.

Responsibilities

• Assess current communication technologies, patterns, recurring problems, and capability gaps across engineering teams.
• Define evaluation criteria and use load tests and benchmarks under representative traffic and failure conditions to select and maintain a Core Collection of recommended communication technologies, patterns, and practices.
• Document recommended technologies, valid alternatives, and exceptional cases through decision guides, reference architectures, architecture decision records (ADRs), and tuning and troubleshooting guides.
• Build and maintain validated reference implementations, starter templates, and shared libraries for common communication use cases.
• Guide the choice between synchronous request/response, asynchronous messaging, event streaming, and other communication patterns.
• Establish practices for delivery guarantees, ordering, retries, deduplication, and idempotency.
• Guide backpressure, buffering, batching, flow control, and consumer concurrency design.
• Optimize throughput, latency, serialization overhead, network utilization, and memory pressure.
• Establish practices for message schemas, data contracts, and backward- and forward-compatible schema evolution.
• Guide partitioning, fan-out, replay, retention, and consumer-group design.
• Establish practices for fault handling, dead-letter processing, observability, and recovery.
• Partner with engineers through pair programming, design and code reviews, troubleshooting, and focused workshops to support consistent adoption of recommended practices.
• Collaborate with Data Storage and Deployment Environments domain leads on cross-domain problems such as consumer lag, memory pressure, distributed tracing, and end-to-end throughput and latency.
• Periodically reassess and update recommendations and documentation as technologies, workloads, and engineering needs change.
• Track adoption and engineering outcomes against established baselines and report results to engineering leadership.
• Perform other duties as assigned.

• Bachelor’s degree in Computer Science, Engineering, Mathematics, Physical Sciences, or a related technical field.
• 10+ years of professional experience designing, implementing, and troubleshooting production software systems, including significant experience with distributed communication systems.
• Deep production experience with one or more message brokers or event streaming platforms (e.g., Kafka, RabbitMQ, ActiveMQ, MQTT).
• Experience with one or more RPC or low-level messaging frameworks (e.g., gRPC, ZeroMQ).
• Strong understanding of delivery semantics, ordering, idempotency, backpressure, and fault handling in distributed systems.
• Experience with serialization formats and schema evolution (e.g., Protocol Buffers, Avro, FlatBuffers, JSON Schema).
• Demonstrated ability to load test and profile communication paths, analyze results, and turn findings into actionable recommendations.
• Experience evaluating technical alternatives and clearly articulating their tradeoffs.
• Proficiency in at least one backend language such as C++, Python, Go, Rust, Java, or TypeScript/Node.js.
• Comfortable working in Linux development environments.
• Strong technical writing, mentoring, and cross-team communication skills, with the ability to influence engineering practices through technical credibility.
• Ability to obtain and maintain a DoD security clearance.

Candidates with the following qualifications will be preferred:

• Experience leading technology standardization, technical governance, or internal platform/guidance efforts across multiple teams.
• Experience with high-throughput or low-latency data pipelines, including network and memory performance tuning.
• Experience with distributed tracing and observability tooling (e.g., OpenTelemetry).
• Experience running messaging systems in containerized or Kubernetes environments.
• Familiarity with mission data workflows or sensor processing architectures.
• Active DoD Secret clearance or higher.

*Resumes, Cover Letters, and Applications which are generated by AI will not be considered for employment.

Colorado Residents:

In any materials you submit, you may redact or remove age-identifying information such as age, date of birth, or dates of school attendance or graduation. You will not be penalized for redacting or removing this information

SciTec offers a highly competitive salary and benefits package, including:

• 4% Safe Harbor 401(k) match
• 100% company paid HSA Medical insurance, with a choice of 2 buy-up options
• 80% company paid Dental insurance
• 100% company paid Vision insurance
• 100% company paid Life insurance
• 100% company paid Long-term Disability insurance
• Short-term Disability insurance
• Annual Profit-Sharing Plan
• Discretionary Performance Bonus
• Paid Parental Leave
• Generous Paid Time Off, including Holiday, Vacation, and Sick Pay
• Flexible Work Hours

The pay range for this position is $164,000 - $218,000 / year. SciTec considers several factors when extending an offer of employment, including but not limited to the role and associated responsibilities, a candidate's work experience, education/training, and key skills. This is not a guarantee of compensation.

SciTec is proud to be an Equal Opportunity employer. VET/Disabled.