Senior Analytics Engineer

Carbon Arc, Inc.; New York, Northern (NY, KY); Hybrid; USD 120,000 - 160,000; Full time

Benefits offered by this job

Remote work options

Healthcare benefits (medical, dental,/

Paid time off

Parental leave

Competitive team culture

Job summary

Carbon Arc, Inc. is seeking a Senior Analytics Engineer to own and advance critical data pipelines and analytics infrastructure. You will contribute to ML and AI products, building scalable systems for data quality, ontology enrichment, and knowledge graph-driven apps.

You’ll work across the full data stack, collaborating with Engineering, Product, and Insights to deliver production-grade solutions at petabyte scale.

Qualifications

• 4+ years of production data pipelines experience.
• BS or MS in CS, DS, or related field.
• Proficient in Python and SQL; PySpark experience.
• Experience with lakehouse architectures and data quality.

Responsibilities

• Own one or more datasets end-to-end through data pipeline (extraction, cleaning, transformation, delivery).
• Build and maintain EXD tables for client-facing APIs and dashboards.
• Design ontology mapping pipelines across sources; maintain mapping infra at scale.
• Develop automated EDA and confidence scoring to evaluate data quality.
• Contribute to knowledge graphs and platform tooling for velocity.
• Collaborate with Engineering, Product, and Insights to translate requirements.
• Support cross-functional data infrastructure migration and cost optimization.

Skills

Python

SQL

Data pipelines

Distributed computing

Documentation

Technical communication

Education

BS in Computer Science or related

MS in CS or Data Science

Tools

DBT

Airflow

Apache Iceberg

Parquet

S3

Trino/Spark

Job description

The Research & Intelligence team builds and maintains the analytics data platform that powers Carbon Arc’s products — from raw data ingestion and entity mapping to certified data pipelines, knowledge graphs, and AI-powered applications. We own the full lifecycle of alternative data: cleaning, transforming, enriching, scoring, and serving it to clients through APIs, dashboards, and intelligent search.

We are a high-trust, high-impact team that takes end-to-end ownership of complex data problems, working across petabyte-scale datasets in a fast-paced, collaborative environment.

We’re looking for a Senior Analytics Engineer to own and advance critical data pipelines, build production-grade analytics infrastructure, and contribute to the ML and AI products that differentiate Carbon Arc. You will work across the full data stack — from raw extraction through certified delivery — designing scalable systems that ensure data quality, enrich our ontology, and power knowledge graph-driven applications. This role blends data engineering rigor with analytical depth and product sensibility.

What You'll Do

• Own one or more alternative datasets end-to-end through our data pipeline (extraction, cleaning, transformation, certification, and delivery), ensuring data quality and timeliness across petabyte-scale workloads.
• Build and maintain Entity Explorer Data (EXD) tables that serve as the foundation for client-facing APIs, dashboards, and analytical products, including demographic, financial, and behavioral metrics at varying entity and temporal granularities.
• Design and execute ontology mapping pipelines — resolving entities (brands, companies, products, locations) across disparate data sources using deterministic and probabilistic methods, and maintaining mapping infrastructure at scale.
• Develop and run automated EDA and confidence scoring pipelines to evaluate data quality, detect anomalies, and quantify panel representativeness across datasets and time periods.
• Contribute to the construction and maintenance of the Carbon Arc knowledge graphs.
• Build and extend internal tooling and infrastructure - including report templates, monitoring frameworks, bulk data generation, and Pydantic-based validation models - to improve team velocity and pipeline reliability.
• Collaborate with Engineering, Product, and Insights teams to translate business requirements into scalable data solutions, and participate in cross-functional initiatives such as data infrastructure migration, metadata management, and platform cost optimization.

What You’ll Bring

• 4+ years of experience building and maintaining production data pipelines and analytics infrastructure at scale.
• BS or MS in Computer Science, Data Science, Statistics, Engineering, or a related quantitative discipline.
• Deep proficiency in Python and SQL, with hands-on experience writing PySpark and working with distributed query engines (Trino, Spark).
• Strong experience with columnar data formats and lakehouse architectures (Apache Iceberg, Parquet, S3-backed data lakes).
• Familiarity with data transformation frameworks such as DBT and pipeline orchestration tools (Airflow, or equivalent).
• Experience with entity resolution, ontology design, or semantic data modeling across structured and semi-structured datasets.
• Demonstrated ability to take ambiguous data problems from scoping through production deployment with minimal oversight.
• Clear and precise technical communication, with a track record of strong documentation and cross-functional collaboration.

Nice to Have

• Hands-on experience with graph databases (Neo4j) and knowledge graph construction, including ontology-driven node/relationship modeling.
• Familiarity with GenAI tooling and LLM-based applications, including RAG architectures, prompt engineering, and tool-use frameworks (e.g., MCP).
• Experience with StarRocks, ClickHouse, or other OLAP engines for high-performance analytical queries.
• Experience building internal developer tools, CLI applications, or data quality monitoring dashboards.

What We Offer

• Fully paid healthcare benefits (Medical, Dental, Vision)
• Remote work options
• Paid time off
• Generous parental and family leave policies
• Strong work culture focused on integrity, transparency and excellence.

Location

Carbon Arc is headquartered in New York City and operates as a remote-first company. Team members are expected to travel for in-person onsite gatherings at least one week per quarter to collaborate, plan, and connect as a team.

Equal Opportunity Employer

Carbon Arc is an Equal Opportunity Employer and is committed to fair and equitable hiring practices. All hiring decisions are based on strategic business needs, job requirements, and individual qualifications. All candidates are considered without regard to race, color, religion, sex, sexual orientation, gender identity or expression, national origin, age, disability, genetic information, marital status, pregnancy, veteran or military status, citizenship status, or any other characteristic protected by applicable federal, state, or local law.

Accommodations

Carbon Arc is committed to providing reasonable accommodations throughout the hiring process for qualified individuals with disabilities. If you require an accommodation to participate in the application or interview process, please contact us at [e-mail] .