Senior Data Engineer - Data Sourcing

Revelio Labs; New York (NY); Remote; USD 120,000 - 200,000; Full time

Job summary

An innovative firm is seeking a Senior Data Engineer to lead data sourcing and engineering efforts. This role involves designing web crawlers, managing data ingestion, and integrating proprietary models to enhance data quality. Join a dynamic team dedicated to transforming public data into valuable workforce insights. If you're passionate about data and technology, this is your chance to make a significant impact in the labor market analysis space.

Benefits

Medical Insurance

Dental Insurance

Vision Insurance

Retirement Plan

Equity Program

Qualifications

• Proven expertise in data sourcing, web scraping, and data engineering.
• Experience building large-scale, high-volume web scrapers.

Responsibilities

• Oversee and own the full lineage of data sources and feeds.
• Design and implement web crawlers to source public data.

Skills

Python

TypeScript

Web Scraping

Data Engineering

Bot Prevention Services

Web Technologies

Tools

ETL/ELT Pipelines

Cloud Monitoring Tools

Big Data Processing Tools (e.g., Spark)

CI/CD Practices

Job description

Join to apply for the Senior Data Engineer - Data Sourcing role at Revelio Labs

Join to apply for the Senior Data Engineer - Data Sourcing role at Revelio Labs

Get AI-powered advice on this job and more exclusive features.

Who We Are

Revelio Labs provides workforce intelligence. We absorb and standardize hundreds of millions of public employment records to create the world’s first universal HR database, allowing us to see current workforce composition and trends of any company. Our customers include investors, corporate strategists, HR teams, and governments.

Who We Are

Revelio Labs provides workforce intelligence. We absorb and standardize hundreds of millions of public employment records to create the world’s first universal HR database, allowing us to see current workforce composition and trends of any company. Our customers include investors, corporate strategists, HR teams, and governments.

What We’re Looking For

Revelio Labs is looking for a data acquisition specialist who has proven expertise in data sourcing, web scraping, and data engineering. You will oversee and own the full lineage of data sources and feeds, from scraping and parsing to ingestion and model integration.

We are looking for someone who can design and implement web crawlers to source public data that will generate powerful insights on the labor market, integrate proprietary models to standardize and augment data sources, and increase the resiliency of existing data streams, as well as build a generalized framework to automate data acquisition, ingestion, and augmentation.

Required

Experience and Skills:

• Strong knowledge of Python and TypeScript
• Experience building large-scale, high-volume web scrapers
• Experience with bot prevention services and reverse engineering web applications
• Deep understanding of web technologies, frameworks, and network protocols (HTML, JavaScript, HTTP, etc)
  
  

Preferred

• Experience building and maintaining ETL/ELT pipelines
• Experience with cloud monitoring and administration tools
• Familiarity with big data processing tools (e.g. Spark)
• Knowledge of data warehouse maintenance best practices, including data wrangling, model integration, anomaly detection, and documentation
• Knowledge of common CI/CD practices, including continuous build/test/deploy automation
  
  

Location:

Our offices are based in New York City, but the position can be done remotely.

Salary

The pay range for this position in New York City is $120,000 - $200,000 per year. The salary range for performing this role outside of New York City may differ. Base pay offered may vary depending on job-related knowledge, skills, and experience. Additionally, you may be eligible to participate in our company’s equity program, plus benefits, including medical, dental, vision, retirement, and other. The range above is for the expectations as laid out in the job description, however we are often open to a wide variety of profiles, and recognize that the person we hire may be more senior or have different experience than this job description as posted. If that ends up being the case, the updated salary range will be communicated to you as a candidate.

Why You Should Work With Us

We are putting public data to incredible use, providing unparalleled insight into the workforces of companies and industries. You’ll work with a strong and innovative team of engineers who have created the most comprehensive set of labor market data available, and who will continue to discover valuable information in all corners of the public web.

How You Should Reach Us

Please email your resume to [e-mail] as a PDF file. Please include your GitHub and highlight any projects that you’ve worked on that may be relevant.

Apply

Please find our CPRA Job Applicant Privacy Notice here.

Seniority level

• Seniority level

  Mid-Senior level

Employment type

• Employment type

  Full-time

Job function

• Job function

  Information Technology

• Industries

  Technology, Information and Internet

Referrals increase your chances of interviewing at Revelio Labs by 2x

Get notified about new Senior Data Engineer jobs in New York, NY.

New York, NY $100,000.00-$200,000.00 2 months ago

New York, NY $125,000.00-$135,000.00 2 weeks ago

New York, NY $153,000.00-$210,000.00 4 months ago

New York, NY $130,000.00-$140,000.00 1 month ago

Senior Data Solutions Developer - Private Markets (VP)

Sr. Software Engineer for Product Support

Senior Software Engineer - Sell-Side Trading

New York, NY $160,000.00-$240,000.00 4 days ago

Senior Engineer Identity & Access Management

Senior Software Engineer, Credit Cards & Banking

Azure Data Architect - Remote With Travel

Data Engineer Manager with Big Data Experience

New York City Metropolitan Area $185,000.00-$250,000.00 1 week ago

Jersey City, NJ $150,000.00-$205,000.00 2 months ago

Azure Cloud Data Architect - Purchase NY, Morristown NJ, Charlotte NC (Hybrid)

New York City Metropolitan Area 1 month ago

New York City Metropolitan Area 1 month ago

New York, NY $158,000.00-$198,000.00 3 days ago

New York City Metropolitan Area 3 weeks ago

Senior / Staff Software Engineer - Computational Chemistry / Molecular Dynamics

Data Architect (IBM Planning Analytics) - Hybrid

Lake Success, NY $105,400.00-$183,000.00 3 weeks ago

We’re unlocking community knowledge in a new way. Experts add insights directly into each article, started with the help of AI.