Manager, Software Development (Content Integration & Data Pipelines)

Remote; San Francisco (CA); Remote; USD 104,000 - 118,000; Full time

Benefits offered by this job

Fully remote work

RRSP matching

Health, dental, vision

Vacation, personal days, sick leave

No internal Fridays meetings

Job summary

Partner Company seeks a Manager of Software Development (Content Integration & Data Pipelines) to lead a technical team in building scalable content collection and matching systems. The role focuses on guiding architecture for web crawling, data ingestion, and mention matching, while collaborating with product owners to prioritize the technical roadmap.

You will oversee reliability, performance metrics, and on-call practices, while advancing AI-enabled workflows and analytics.

Qualifications

• 7+ years of software development experience, including web crawling, data ingestion, or large-scale data pipelines.

Responsibilities

• Lead, mentor, and develop a team of software developers, establishing objectives and fostering accountability.

Skills

Java

Distributed systems

Cloud platforms

Docker

Kubernetes

Kafka

CI/CD

Elasticsearch

APIs integration

NLP/LLMs

Agile / JIRA / Confluence

Data pipelines

Education

Bachelor's degree in CS or related

Tools

Kafka

Docker

Kubernetes

JIRA

Confluence

Elasticsearch

Job description

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Manager, Software Development (Content Integration & Data Pipelines) based in Canada.

In this leadership role, you will guide a software development team responsible for collecting, integrating, and matching content from across the digital landscape. You will shape the architecture behind scalable media monitoring and social listening systems, ensuring clients receive timely, accurate, and relevant insights. You will also contribute to the development of AI-powered applications, intelligent tools, and agentic workflows. Working closely with cross-functional teams, you will translate business priorities into technical solutions that deliver measurable value. This opportunity combines hands-on technical leadership, people development, and complex data engineering in a collaborative, fully remote environment.

Accountabilities

• Lead, mentor, and develop a team of software developers, establishing clear objectives and fostering accountability, collaboration, continuous improvement, and technical excellence.

• Define the technical strategy and architecture for web crawling, content collection, data ingestion, and mention-matching systems.

• Collaborate with Product Owners to translate customer needs and business objectives into a prioritized technical roadmap.

• Ensure data pipelines and collection systems remain scalable, reliable, efficient, and cost-effective as content volumes and source coverage expand.

• Establish and monitor performance metrics covering source coverage, mention accuracy, processing latency, uptime, and overall data quality.

• Oversee incident response and on-call practices, lead root-cause investigations, and implement improvements that strengthen production reliability.

• Stay involved in technical execution through architecture discussions, design and code reviews, and hands-on troubleshooting when needed.

• Partner with Customer Success to investigate and resolve client-reported gaps in media coverage and missed mentions.

• Ensure data collection practices comply with website crawling directives, platform terms of service, copyright and licensing requirements, and applicable privacy legislation, consulting Legal when appropriate.

• Assess emerging technologies, AI techniques, and data providers to improve content coverage, relevance, and system capabilities.

• Manage relationships with third-party data vendors and platform API partners in collaboration with Content Partnerships.

• Support the evolution of AI-enabled workflows and applications that use comprehensive, timely content data to generate valuable insights and alerts.

Requirements:

• 7+ years of software development experience, including work with web crawling, data ingestion, or large-scale data pipelines.

• At least 2 years of experience managing or leading software development teams.

• A bachelor's degree in Computer Science or a related discipline, or equivalent practical experience.

• Strong technical expertise in Java or a comparable type-safe programming language.

• Experience designing distributed systems, including message queues or streaming technologies such as Kafka.

• Familiarity with cloud platforms such as AWS, GCP, or Azure, as well as containerization technologies like Docker and Kubernetes and modern CI/CD practices, is an asset.

• Knowledge of search technologies such as Elasticsearch or Solr, along with Boolean query logic.

• Experience integrating social media platform APIs and working with third-party data providers.

• A solid understanding of the legal, ethical, and privacy considerations associated with web and social data collection.

• Experience working in Agile development environments and using tools such as JIRA and Confluence.

• Strong communication and collaboration skills, with the ability to explain complex technical concepts to both technical and non-technical stakeholders.

• A data-driven approach to leadership, using metrics to guide technical decisions, assess performance, and measure team outcomes.

• A supportive leadership style that balances delivery expectations with team development, engagement, and well-being.

• Experience with natural language processing (NLP), large language models (LLMs), entity recognition, or sentiment analysis is an asset.

• Experience in PR technology, media monitoring, social listening, or the newswire industry is an advantage.

Benefits:

• Annual salary: CAD $145,000–$165,000, with the final offer based on experience, skills, and internal equity.

• Fully remote work environment.

• Collaborative workplace culture supported by tools and practices that enable effective teamwork.

• Health, dental, and vision insurance coverage.

• Registered Retirement Savings Plan (RRSP) matching.

• Life insurance.

• Employee Assistance Program (EAP).

• Career development and progression opportunities.

• Paid vacation, personal days, and sick leave.

• A week off between Christmas and New Year's.

• No internal meetings on Fridays, supporting focused work and greater flexibility.