Senior / Principal Software Engineer

Blackbird.AI; United States; Remote; USD 90,000 - 150,000; Full time

Job summary

Join a forward-thinking company as a Senior/Principal Software Engineer, where you'll optimize data pipelines and develop cutting-edge features for a real-time streaming analytics platform. This role offers a unique opportunity to tackle challenging data problems while collaborating with AI research teams. You'll be instrumental in shaping the future of narrative intelligence, ensuring data integrity, and driving innovative solutions that impact businesses and society. With a focus on continuous learning and professional growth, this is the perfect place for passionate individuals ready to make a difference.

Benefits

Competitive compensation package

401(k) and equity

Comprehensive health benefits

Generous vacation policy

Flexible work environment

Professional development stipend

Qualifications

• 5+ years in data engineering and building data platforms.
• 2+ years in Machine Learning with a successful track record.
• Expertise in building optimized processing pipelines.

Responsibilities

• Design and implement real-time distributed data processing systems.
• Lead development of new features and capabilities for the platform.
• Conduct performance analysis and make strategic decisions.

Skills

Python

Data Engineering

Machine Learning

ETL Processes

Database Optimization

Project Management

Education

Bachelor's degree in Computer Science

Tools

PostgresSQL

ElasticSearch

MongoDB

Redis

Druid

Kafka

Airflow

Apache Spark

Job description

BLACKBIRD.AI protects organizations from narrative attacks that cause financial and reputational harm. Our Constellation AI-driven Narrative Intelligence Platform – identifies key narratives that impact an organization/industry, the influence behind them, the networks they touch, the anomalous bot behavior that scales them, and the cohorts and communities that connect them. This information enables organizations to proactively understand narrative threats as they scale and become harmful for better strategic decision-making. A diverse team of AI experts, threat intelligence analysts, and national security professionals founded Blackbird.AI to defend information integrity and fight a new class of narrative threats.

As a Senior/Principal Software Engineer reporting to the Head of Engineering, you will not only optimize data pipelines for our advanced real-time streaming cloud-hosted analytics platform but also lead the development of new features and capabilities. Your role will be pivotal in shaping the future of our platform, offering the chance to solve some of the most challenging data problems in the industry. You will be the key driver of our collection, analysis, and visualization processes and will play a pivotal role in shaping the future of our platform.

This Engineer will also work with our AI research and development team and be responsible for a broad range of cutting-edge applications in machine learning, addressing critical challenges impacting businesses, governments, and society.

This Engineer’s responsibilities will also encompass writing ETL processes, designing database systems, and developing tools for query and analytic processing, all focusing on real-time streaming applications. Additionally, you will elevate performance tradeoffs, own the database architecture, and lead build automation, continuous integration, and deployment efforts while ensuring adherence to security requirements.

As a Senior/Principal Software Engineer, you’ll have the chance to:

• Design and implement real time distributed data processing systems analyzing public data and detecting emergent threats.
• Focus on research and development in information extraction, multilingual NLP, automated summarization, graph network analysis, Computer visions, and LLMs (Large Language Models).
• Oversee the gathering and annotating of large custom datasets for tasks like text or vision classification, unsupervised pre-training, translation, tagging, and related challenges.
• Apply your knowledge of machine learning performance evaluation techniques, recognizing failure modes and limitations in the process.
• Lead and manage the development and optimization of ETL processes for various data formats from social media, news, and web sources.
• Design and implement robust database systems and develop tools for query and analytic processing, focusing on real-time streaming applications.
• Conduct performance analysis and empirical studies, making strategic decisions on tradeoffs (e.g., cost vs. throughput/latency).
• Develop, manage, and oversee the database architecture for our real-time streaming cloud-hosted analytics platform.
• Spearhead build automation, continuous integration, deployment, and performance optimization efforts, upholding our strict security requirements.
• Drive the project management, leading the development of new features and capabilities.
• Design test suites and implement inline instrumentation to ensure data correctness.
• Design secure and scalable REST APIs.

What you’ll bring:

• Bachelor's degree in Computer Science or a related field.
• Minimum of 5 years of experience in data engineering and building data platforms.
• Minimum of 2 years of professional experience in Machine Learning or a closely related field, demonstrating a track record of successfully delivering novel research and/or product offerings.
• Demonstrated success in deploying cloud and SaaS model products, with expertise in building optimized processing pipelines for streaming analytics applications.
• Proficiency in databases and query optimization (PostgresSQL, ElasticSearch, MongoDB, Redis, Druid), including experience with NoSQL and graph databases.
• Experience in horizontally scaling databases.
• Expertise in Kafka and Airflow, with a strong understanding of runtime profiling tools.
• Experience in big data processing systems similar to Apache Spark, Flink, Beam or similar.
• Strong skills in build automation, continuous integration, and deployment (CI/CD) tools.
• Expert-level Python coding skills.

• Project management skills and the ability to lead development initiatives.

Desirable Skills:

• Familiarity with AWS and other cloud services.
• Experience with MLOps frameworks like Kubeflow, MLFlow, or similar.
• Technical background or experience in AI/ML deployments.
• Ability to create unique datasets for scientific analysis or benchmarking.
• Capability to contribute to system architecture enhancements to improve scalability, testability, and robustness.
• Experience with generative models for generating fake text or images.
• A history of top performance in online competitions or hackathons, such as Kaggle.
• Published research in areas related to machine learning, NLP, Computer Vision, or its applications.
• Contributions to open-source machine learning projects or related initiatives, including platforms like Github and HuggingFace.

We’ve outlined specific skills, experience, and requirements for this position, but don’t stress if you don’t meet every single one. Our Talent Team is dedicated to discovering exceptional individuals, and they might identify a relevant aspect of your background that suits this role or another opportunity within Blackbird.AI.

If you have passion for the role, please still apply.

What’s in it for you:

Blackbird.AI is embarking on an exciting growth journey with numerous opportunities for career development within the company. You will join a nurturing, inclusive, and experienced team.

Join us as we soar to new heights!

Values:

At Blackbird.AI, our core values shape how we work and make decisions. Our values inspire us to be authentic and continue improving.

We embrace a strong sense of responsibility to society, recognizing the vital role our services play in empowering governments, communities, and individuals to foster critical thinking.. We believe in integrating personal and professional lives with societal needs, emphasizing the importance of creating an environment that attracts top talent and provides substantial growth opportunities. We are motivated by the potential of science and technology to impact humanity positively

Why you’ll love working here:

• Competitive compensation package, 401(k), and equity - everyone has a stake in our growth!
• Comprehensive health benefits for you and your loved ones, including wellness days and monthly wellness reimbursements - an apple a day doesn’t always keep the doctor away!
• Generous vacation policy, encouraging you to take the time you need - we trust you to strike the right work/life balance!
• A flexible work environment with opportunities to collaborate with your team in person - you can have it all!
• Inclusion and Impact - soar to new heights!
• Professional development stipend - never stop learning!

Apply Today

Equal Opportunity Employer