Data Engineer: Market Intelligence Pipelines (Remote)

Crux; New York (NY); Remote; USD 130,000 - 180,000; Full time

Job summary

Crux is hiring a Data Engineer to build and own the Market Intelligence data foundation. You will create and maintain production data pipelines, ingest survey responses, pricing series, and forecasts, and ensure data quality for publications and client deliverables.

You will automate survey processes, design scalable data workflows, and enable reporting through robust data infrastructure while collaborating with researchers and product teams across a remote-first, US-and-Canada-friendly setup.

Qualifications

• 3+ years of experience building and maintaining production data pipelines in high-growth, fast-paced environments.
• Fluent in Python and SQL, with the ability to write, test, and debug production code and complex queries.
• Proficient in cloud data warehousing, including schema design, incremental data loading, query optimization, and cost management.
• Experience with extraction, transformation, loading, scheduling, and monitoring.
• Knowledge of established data-quality practices, including input validation, automated testing, monitoring, documentation, version control, and reproducible outputs.
• Skilled at translating business or research needs into clearly scoped technical requirements, deliverables, and implementation plans.
• Demonstrated ability to communicate technical tradeoffs, project status, data limitations, and delivery risks to technical and nontechnical stakeholders.
• Hands-on experience with AI-enabled tools supporting software development, data analysis, testing, documentation, or workflow automation.
• Sound technical judgment when evaluating datasets and engineering approaches against downstream reporting, research, or product requirements.

Responsibilities

• Own the Market Intelligence data foundation. Build and maintain the pipelines our market intelligence work runs on: survey responses, pricing series, volume and capacity forecasts, buyer participation. Extraction, transformation, scheduling, monitoring, the whole path.
• Automate our survey processes end to end. Ingesting, cleaning, joining - a lot of this is ad hoc today. Create a system to scale our survey program.
• Get data where it needs to be. Support the reporting layer so findings are visible and plugged into dashboards and publications.
• Turn recurring questions into repeatable queries as bespoke research scales.
• Firm up Crux's data practices by validating inputs, documenting assumptions, version control, reproducible outputs, and identifying new use cases across teams.

Skills

Python

SQL

Data pipelines

Cloud data warehousing

ETL processes

Data quality

Automation

Job description

Crux is hiring a Data Engineer to build and own the Market Intelligence data foundation. You will create and maintain production data pipelines, ingest survey responses, pricing series, and forecasts, and ensure data quality for publications and client deliverables.

You will automate survey processes, design scalable data workflows, and enable reporting through robust data infrastructure while collaborating with researchers and product teams across a remote-first, US-and-Canada-friendly setup.