Back end Software Engineer (web scraping/data acquisition)

Sapient Search; United States; On-site; USD 120,000 - 180,000; Full time

Job summary

Sapient Search seeks a seasoned software engineer for its data ingestion team in the United States. You will design, implement, and maintain highly transactional services that move data from diverse sources into our systems.

The role requires strong TypeScript/NodeJS, Docker (and Docker Compose), AWS, and modern testing with Jest or Vitest, plus experience building robust, well-typed code and scalable data pipelines.

Qualifications

• 10+ years of software development experience.
• 2+ years with TypeScript/NodeJS and well-typed, OO code.
• Experience running container-based applications (Docker, Docker Compose).
• 2+ years leveraging HTTP knowledge and clients (got, curl, axios).
• Proficient in unit testing with Jest or Vitest.
• Hands-on experience building apps on AWS hosted solutions.

Responsibilities

• Envision, create, test, and maintain code for highly transactional services.
• Design robust and scalable data acquisition strategies for data ingestion pipelines.
• Ensure efficient and reliable flow of data from diverse sources into systems.

Skills

Software development experience

TypeScript/NodeJS

HTTP clients (axios, got, curl)

Unit testing (Jest, Vitest)

AWS experience

Linux knowledge

LLMs for engineering velocity

Education

BSc in Computer Science or equivalent

5-10 years software engineering experience

Tools

Docker

Docker Compose

HTTP clients (axios)

Jest

Vitest

TypeScript

NodeJS

AWS

Redis/ElastiCache

DynamoDB

Datadog

New Relic

Playwright

Puppeteer

Cypress

Selenium

DI containers

Lambda

SQS

BullMQ

GitHub Actions

CDK

Terraform

CloudFormation

React

Vue

Serverless

Job description

Size: late stage startup, 400 employees, heavily revenue positive

As a member of our data ingestion team, you will envision, create, test, and maintain code for several highly transactional services. Your role will involve designing robust and scalable data acquisition strategies, ensuring the efficient and reliable flow of data from diverse sources into our systems.

Technical Qualifications

• Must have 10 years of Software Development experience
• Must have 2+ years of experience with TypeScript/NodeJS and deep understanding of writing well-typed, object-oriented code
• Must have experience running container based applications (Docker, Docker Compose)
• Must have 2+ years experience leveraging HTTP knowledge and client skills, e.g. got, cURL, axios
• Must be highly proficient in unit testing methodologies, e.g. with Jest or Vitest
• Must have hands on experience coding applications that run and reside on AWS hosted solutions

Related Qualifications

• Experience with Dependency Injection/DI containers
• Knowledge about best practices relative to end-to-end testing with e.g. Playwright, Puppeteer, Cypress, Selenium
• Experience with Redis/Elasticache
• Experience with DynamoDB
• Broad Linux knowledge
• Hands on experience setting up Monitoring/observability platforms, e.g. Datadog, New Relic
• Must be familiar with using LLMs to increase engineering velocity while still maintaining high coding standards and developer-friendly code

Nice to haves

• Experience with man-in-the-middle techniques (Proxyman, HTTP Toolkit, Wireshark), reverse engineering, understanding decompiled code
• Has utilized and written Serverless functions (Lambda)
• Knowledgeable about Queueing systems, e.g. SQS, BullMQ
• Familiar with GitHub Actions
• Has experience with Domain Driven Design/Clean Architecture
• Has experience with Java/Android Studio
• Worked with Front end frameworks, e.g. React, Vue
• Has hand rolled infrastructure as code, e.g. CDK, SAM, CloudFormation, Terraform

Education

• Minimum of 5-10 years of experience in a software engineering role
• Bachelor of Science Degree in Computer Science or commensurate experience

Core Qualifications

• Assist team members with problem-solving complex use cases and systems; while leading technical change and transformation in parallel.
• Must be passionate about creating solutions, and solving problems – in the right way, at the right time, and for the right reasons.
• Must be teachable, give and receive feedback, and demonstrate success in their discipline on a consistent and transparent basis.