Senior Dataflow Systems Engineer - Cloud & DevOps Leader

Avid Technology Professionals; Maryland; On-site; USD 120,000 - 180,000; Full time

Job summary

Avid Technology Professionals seeks a lead dataflow architect to design, integrate, and sustain complex enterprise data pipelines across large-scale cloud and distributed environments.

Responsibilities include optimizing Linux and Cloudera Hadoop infrastructures, ensuring data governance, monitoring, and automation, while providing DevOps leadership in Agile/Scrum contexts to advance mission-critical analytics initiatives.

Responsibilities

• Lead the design, integration, and sustainment of complex enterprise dataflow architectures supporting mission-critical analytics, ensuring reliable ingestion, processing, routing, and delivery across large-scale cloud and distributed environments.
• Manage and optimize high-availability Linux and Cloudera Hadoop cloud infrastructures, including virtual machines, storage containers, and enterprise data repositories, to support thousands of secure operational dataflows.
• Engineer and maintain end-to-end dataflow solutions using technologies, and related processing systems, ensuring schema compatibility, metadata integrity, and proper data governance throughout the lifecycle.
• Develop and implement monitoring, troubleshooting, and automation strategies utilizing system instrumentation, cron scripting, logging, and operational monitoring tools to improve system continuity, performance, and incident response capabilities.
• Provide technical leadership for DevOps and Agile/Scrum initiatives, including task prioritization, workflow modernization, requirements management, and cross-functional coordination to improve operational efficiency and team performance.
• Ensure compliance with enterprise security, classification, and data handling requirements by applying deep knowledge of intelligence community governance policies, access controls, and secure data routing methodologies.
• Coordinate across engineering, operations, analytics, and stakeholder organizations to deploy new capabilities, transition systems into production, establish standard operating procedures, and support enterprise-wide mission objectives through scalable and resilient dataflow solutions.

Job description

Avid Technology Professionals seeks a lead dataflow architect to design, integrate, and sustain complex enterprise data pipelines across large-scale cloud and distributed environments.

Responsibilities include optimizing Linux and Cloudera Hadoop infrastructures, ensuring data governance, monitoring, and automation, while providing DevOps leadership in Agile/Scrum contexts to advance mission-critical analytics initiatives.