Manager, Software Development (Content Integration & Data Pipelines)

Jobgether; United States; Remote; USD 104,000 - 118,000; Full time

Benefits offered by this job

Fully remote work environment

Health, dental, and vision insurance

RRSP matching

Life insurance

Employee Assistance Program (EAP)

Career development and progression

Paid vacation, personal days, and sick

Week off between Christmas and NewYear

No internal meetings on Fridays

Job summary

Jobgether is seeking a senior engineering leader to guide a team of developers in building and scaling web crawling, data ingestion, and mention-matching systems. You will define technical strategy, collaborate with product stakeholders, and drive reliable data pipelines while maintaining high quality and security standards.

You will lead architecture discussions, oversee incident response, and partner with Customer Success to resolve client gaps in media coverage.

Qualifications

• 7+ years of software development including web crawling, data ingestion, or large-scale data pipelines.
• 2+ years of experience managing software development teams.
• Bachelor's degree in Computer Science or related discipline, or equivalent practical experience.
• Strong technical expertise in Java or a comparable type-safe programming language.
• Experience designing distributed systems, including message queues or streaming technologies such as Kafka.
• Familiarity with cloud platforms such as AWS, GCP, or Azure, containerization technologies like Docker and Kubernetes, and modern CI/CD practices.
• Knowledge of search technologies such as Elasticsearch or Solr, along with Boolean query logic.
• Experience integrating social media platform APIs and working with third-party data providers.
• A solid understanding of legal, ethical, and privacy considerations associated with web and social data collection.
• Experience working in Agile development environments and using tools such as JIRA and Confluence.
• Strong communication and collaboration skills, with the ability to explain complex technical concepts to both technical and non-technical stakeholders.
• A data-driven approach to leadership, using metrics to guide technical decisions, assess performance, and measure team outcomes.
• A supportive leadership style that balances delivery expectations with team development, engagement, and well-being.
• Experience with NLP, large language models (LLMs), entity recognition, or sentiment analysis is an asset.
• Experience in PR technology, media monitoring, social listening, or the newswire industry is an advantage.

Responsibilities

• Lead, mentor, and develop a team of software developers, establishing objectives and fostering accountability and excellence.
• Define the technical strategy and architecture for web crawling, data ingestion, and mention-matching systems.
• Collaborate with Product Owners to translate needs into a prioritized technical roadmap.
• Ensure scalable, reliable, and cost-effective data pipelines as content grows.
• Establish and monitor performance metrics across coverage, accuracy, latency, uptime, and quality.
• Oversee incident response and on-call practices; drive root-cause analysis and improvements.
• Participate in architecture discussions, reviews, and hands-on troubleshooting.
• Partner with Customer Success to address client gaps in media coverage.
• Ensure data collection complies with crawling directives, terms of service, copyright, licensing, and privacy laws; consult Legal when needed.
• Assess emerging technologies and data providers to improve coverage and capabilities.
• Manage relationships with third-party data vendors and API partners with Content Partnerships.
• Support evolution of AI-enabled workflows using comprehensive content data for insights and alerts.

Skills

Leadership

Team management

Java

Distributed systems

Cloud platforms

CI/CD

Elasticsearch/Solr

NLP/LLMs

Education

Bachelor's degree in CS or related

Tools

Kafka

Docker

Kubernetes

Elasticsearch

Solr

AWS

GCP

Azure

Job description

Accountabilities

- Lead, mentor, and develop a team of software developers, establishing clear objectives and fostering accountability, collaboration, continuous improvement, and technical excellence.

- Define the technical strategy and architecture for web crawling, content collection, data ingestion, and mention-matching systems.

- Collaborate with Product Owners to translate customer needs and business objectives into a prioritized technical roadmap.

- Ensure data pipelines and collection systems remain scalable, reliable, efficient, and cost-effective as content volumes and source coverage expand.

- Establish and monitor performance metrics covering source coverage, mention accuracy, processing latency, uptime, and overall data quality.

- Oversee incident response and on-call practices, lead root-cause investigations, and implement improvements that strengthen production reliability.

- Stay involved in technical execution through architecture discussions, design and code reviews, and hands‑on troubleshooting when needed.

- Partner with Customer Success to investigate and resolve client‑reported gaps in media coverage and missed mentions.

- Ensure data collection practices comply with website crawling directives, platform terms of service, copyright and licensing requirements, and applicable privacy legislation, consulting Legal when appropriate.

- Assess emerging technologies, AI techniques, and data providers to improve content coverage, relevance, and system capabilities.

- Manage relationships with third‑party data vendors and platform API partners in collaboration with Content Partnerships.

- Support the evolution of AI-enabled workflows and applications that use comprehensive, timely content data to generate valuable insights and alerts.

Requirements:

- 7+ years of software development experience, including work with web crawling, data ingestion, or large‑scale data pipelines.

- At least 2 years of experience managing or leading software development teams.

- A bachelor's degree in Computer Science or a related discipline, or equivalent practical experience.

- Strong technical expertise in Java or a comparable type‑safe programming language.

- Experience designing distributed systems, including message queues or streaming technologies such as Kafka.

- Familiarity with cloud platforms such as AWS, GCP, or Azure, as well as containerization technologies like Docker and Kubernetes and modern CI/CD practices, is an asset.

- Knowledge of search technologies such as Elasticsearch or Solr, along with Boolean query logic.

- Experience integrating social media platform APIs and working with third‑party data providers.

- A solid understanding of the legal, ethical, and privacy considerations associated with web and social data collection.

- Experience working in Agile development environments and using tools such as JIRA and Confluence.

- Strong communication and collaboration skills, with the ability to explain complex technical concepts to both technical and non‑technical stakeholders.

- A data‑driven approach to leadership, using metrics to guide technical decisions, assess performance, and measure team outcomes.

- A supportive leadership style that balances delivery expectations with team development, engagement, and well‑being.

- Experience with natural language processing (NLP), large language models (LLMs), entity recognition, or sentiment analysis is an asset.

- Experience in PR technology, media monitoring, social listening, or the newswire industry is an advantage.

Benefits:

- Annual salary: CAD $145,000–$165,000, with the final offer based on experience, skills, and internal equity.

- Fully remote work environment.

- Collaborative workplace culture supported by tools and practices that enable effective teamwork.

- Health, dental, and vision insurance coverage.

- Registered Retirement Savings Plan (RRSP) matching.

- Life insurance.

- Employee Assistance Program (EAP).

- Career development and progression opportunities.

- Paid vacation, personal days, and sick leave.

- A week off between Christmas and New Year's.

- No internal meetings on Fridays, supporting focused work and greater flexibility.