Backend Engineer HighVolume Data Processing

Dreams 2 Reality Recruitment; New York (NY); On-site; USD 220,000 - 300,000; Full time

Benefits offered by this job

Relocation assistance

OPT/H-1B sponsorship

Job summary

Dreams 2 Reality Recruitment in New York seeks a Backend Engineer to build data-processing pipelines that de-identify enterprise data and prepare AI-training datasets. You will own backend systems end-to-end in a fast-growing startup environment.

This role focuses on the processing logic, not traditional data engineering, with opportunities to influence architecture, reliability, and observability. You’ll work onsite in Dumbo, Brooklyn, five days per week.

Qualifications

• 3–10 years of software engineering experience with ownership of production backend systems.
• Experience shipping/operating production software with multi-stage data-processing pipelines.
• Undergraduate STEM degree from a top-30 US institution.
• Strong understanding of distributed and asynchronous backend systems.
• Full-time startup experience with ownership in ambiguous environments.
• Strong proficiency in a general-purpose backend language.
• Willing to work onsite five days per week in Dumbo, Brooklyn, NY.

Responsibilities

• Design, build, and scale multi-stage, high-volume data-processing pipelines that de-identify sensitive enterprise data.
• Own backend systems end-to-end, from architecture and implementation through debugging and production monitoring.
• Develop safeguards to detect silent failures, duplicates, and stale data before delivery.
• Build retries, checkpoints, partial failures, replay, backfills, and rollbacks across workflows.
• Develop internal tools and dashboards to monitor pipeline performance and data quality.
• Partner with ML, full-stack, platform, and security engineers to expand capabilities.
• Improve reliability, observability, and scalability of complex data-processing systems.

Skills

Backend ownership

Production systems

Distributed systems

Multi-stage pipelines

Startup experience

Onsite in NYC

Education

Undergraduate STEM degree (top-30 US)

Tools

Python

AWS

Airflow

AWS Batch

SQS

S3

DynamoDB

PostgreSQL

Job description

Location: Dumbo, Brooklyn, NY
Work Arrangement: Onsite, 5 days per week (required)
Employment Type: Full-time
Base Salary: $220,000–$300,000
Equity: Competitive equity package
Visa: Open to OPT and H-1B transfers; additional sponsorship may be considered
Relocation: Up to $10,000 in relocation assistance

About the Opportunity

A rapidly growing AI and data company is building the data infrastructure that powers next-generation AI models. The company works with proprietary enterprise data, transforming information generated through real-world business operations into de-identified datasets for leading AI research organizations.

Having grown from $0 to a multi-eight-figure annualized run rate within months, the business is scaling rapidly with a lean engineering team of approximately five people.

This is an opportunity to join at an early stage, take meaningful ownership of mission-critical systems, and directly influence the architecture and reliability of a high-impact data-processing platform.

The team is seeking a Backend Engineer to build and evolve the systems that transform sensitive enterprise information into high-quality, AI-training-ready datasets.

The core focus is on the processing logic applied to the data itself, rather than traditional data engineering, ETL, or infrastructure work centered primarily on moving data between systems.

What You’ll Do

• Design, build, and scale multi-stage, high-volume data-processing pipelines that de-identify sensitive enterprise data from collaboration tools, email, cloud storage, code repositories, and project-management systems.
• Own backend systems end-to-end, from architecture and implementation through debugging, correctness validation, production monitoring, and recovery.
• Develop safeguards to detect silent failures, duplicate outputs, stale data, and other data-quality issues before customer delivery.
• Build reliable mechanisms for retries, checkpoints, partial failures, replay, backfills, and rollbacks across asynchronous workflows, batch workers, queues, and object storage.
• Develop internal tools and dashboards to monitor pipeline performance, investigate data quality, and inform release decisions.
• Partner with ML, full-stack, platform, and security engineers to support new data modalities and expand processing capabilities.
• Improve the reliability, observability, and scalability of systems handling complex and sensitive datasets.

What We’re Looking For

Required Qualifications

• 3–10 years of software engineering experience, with direct ownership of production backend systems.
• Demonstrated experience shipping and operating production software, ideally involving multi-stage, high-volume data-processing pipelines.
• Relevant undergraduate STEM degree from a top-30 U.S. institution.
• Strong understanding of distributed and asynchronous backend systems, including:
  • Queues, workers, and concurrency
  • Idempotency and retry strategies
  • Partial failures and fault tolerance
  • Recovery, replay, and data consistency
• Meaningful full-time experience at an early-stage or high-growth startup, with demonstrated ownership, adaptability, and comfort operating in ambiguous environments.
• Strong proficiency in a general-purpose backend programming language.
• Ability and willingness to work onsite five days per week in Dumbo, Brooklyn, NY.

Important: The undergraduate education requirement and five-day onsite requirement are firm. Candidates who do not meet both criteria will not be considered.

Technical Environment

The current technology stack includes:

Python, AWS, Airflow, AWS Batch, SQS, S3, DynamoDB, and PostgreSQL

Python experience is not mandatory. Strong backend engineers with experience in Go, Rust, Java, or other general-purpose languages are encouraged to apply.

Particularly relevant technical experience includes:

• AWS and cloud-based backend systems
• Batch processing and workflow orchestration
• Natural Language Processing (NLP) and Named Entity Recognition (NER)
• Distributed systems and asynchronous processing
• Data quality, correctness, and production reliability
• Internal tooling, monitoring, and observability

The team values engineers who think beyond implementation. Ideal candidates enjoy understanding how data-processing systems behave, building tools to investigate performance and quality, diagnosing complex failures, and solving problems across the entire workflow.

A backend-focused engineer with some full-stack experience can also be a strong fit.

Experience That Stands Out

Candidates may be particularly well suited to this role if they bring:

• Experience at a highly regarded engineering organization combined with meaningful startup experience.
• A clear record of career progression and increasing technical ownership.
• Hands-on AWS experience.
• Experience building or operating batch-processing pipelines and workflow orchestration systems.
• Experience developing internal tools, dashboards, or observability systems for data-processing workflows.
• A strong understanding of production failure modes, data correctness, recovery strategies, and system reliability.
• The ability to take a broad, ambiguous problem and deliver a reliable end-to-end solution.

Who May Not Be a Fit

This position is less suited to candidates whose experience is primarily focused on:

• Traditional data engineering, data platforms, or infrastructure work centered on moving data between systems.
• ETL development using tools such as Databricks, without substantial ownership of the underlying processing logic.
• Frontend-heavy full-stack development without significant backend engineering depth.
• Large-company engineering environments without meaningful early-stage or high-growth startup experience, unless the candidate’s technical background is directly relevant.
• Startup exposure limited mainly to internships or short-term engagements.

Team & Culture

You’ll join a small, high-performing engineering team operating in a fast-moving environment with significant customer demand and substantial room to influence the product and technical direction.

The team emphasizes:

• First-principles problem-solving: Understand the underlying challenge and build the right solution.
• Ownership and accountability: Take responsibility for systems and outcomes from start to finish.
• Bias toward action: Move quickly, experiment, and learn through execution.
• Engineering excellence: Deliver reliable, high-quality work that meets a high technical bar.
• Humility and collaboration: Support teammates, share knowledge, and work together toward common goals.
• Outcome-driven execution: Prioritize meaningful results over activity or output alone.

With a very small engineering team, this role offers significant responsibility, close collaboration, and the opportunity to help shape critical systems as the business scales.

Compensation & Benefits

• Base Salary: $220,000–$300,000
• Equity: Competitive equity package
• Medical, Dental & Vision: 100% covered
• Paid Time Off: Unlimited PTO
• Relocation Assistance: Up to $10,000

Visa & Relocation Support

The company is open to visa transfers, including OPT and H-1B transfers.

Additional visa sponsorship may be considered for exceptional candidates, depending on individual circumstances. Sponsorship is not guaranteed and will be evaluated on a case-by-case basis.

Candidates relocating to New York may be eligible for up to $10,000 in relocation assistance.

Interview Process

The following stages begin after a candidate has been submitted to the hiring team. Agency screening and the initial resume review are not counted as interview stages.

Stage 1 - Introductory Call (30 minutes)

A conversation with the hiring manager covering your background, motivations, experience, and alignment with the role.

Stage 2 - System Design (1 hour)

A distributed-systems and data-processing exercise focused on architecture, technical tradeoffs, scalability, and correctness in high-volume backend pipelines.

Stage 3 - Take-Home Technical Assessment (A few hours)

An independent, practical backend coding exercise evaluating engineering fundamentals and hands-on problem-solving. The engineering team will review the completed submission.

Stage 4 - Final Onsite Interviews (2 hours)

An onsite session at the Dumbo, Brooklyn office with engineering and leadership. The final round evaluates technical depth, problem-solving, and team fit while giving candidates an opportunity to meet the team and learn more about the business.

Travel Support:

Candidates who are not currently based in New York will be flown in and provided with accommodation for the final onsite interview.