# We Built an arxiv Radar That Reads 200 Abstracts Before Breakfast > An ML research team missed a key paper because it landed in cs.CL instead of cs.AI. We built a Struere agent that monitors 6 arxiv categories, reads every abstract, and filters by research relevance. No keywords. No manual skimming. Published: 2026-03-31 Tags: automation, agents, research, arxiv, case-study # The Miss That Started It A computer vision research team at a university lab tracked 6 arxiv categories: cs.CV, cs.AI, cs.LG, cs.RO, stat.ML, and eess.IV. A postdoc named Ren opened arxiv every morning at 8am, scrolled through new submissions, and flagged papers worth reading. About 200 papers per day across those categories. On a Tuesday in February, a group at DeepMind published a paper on geometric priors for 3D scene reconstruction. Directly relevant to Ren's thesis. The paper was posted to cs.CL (Computation and Language) because the authors framed it around multimodal language grounding. Ren never saw it. A colleague at another institution mentioned it two weeks later. The problem was structural. Ren monitored categories. The paper existed outside those categories. No amount of diligence fixes a coverage gap. # The Build: arxiv API + AI Relevance Filter arxiv has a clean API. The OAI-PMH endpoint returns metadata for recent papers. The Atom feed at `export.arxiv.org` gives you title, authors, abstract, categories, and PDF link in structured XML. No scraping. No HTML parsing. Just an HTTP GET. We built four things: 1. **`entity-types/paper.ts`** defines what gets stored 2. **`tools/index.ts`** has a custom tool that fetches the arxiv API and extracts paper metadata 3. **`agents/paper-scout.ts`** reads abstracts and scores relevance 4. **`triggers/daily-scan.ts`** chains the fetch and the filter ## The Entity Type ```typescript import { defineData } from "struere" export default defineData({ name: "Paper", slug: "paper", schema: { arxivId: { type: "string", required: true }, title: { type: "string", required: true }, authors: { type: "string", required: true }, abstract: { type: "string", required: true }, pdfUrl: { type: "string", required: true }, categories: { type: "string", required: true }, relevanceScore: { type: "number", required: true }, relevanceReason: { type: "string", required: true }, publishedDate: { type: "string", required: true }, }, searchFields: ["title", "authors", "abstract", "relevanceReason"], }) ``` ## The Scraper Tool: arxiv Atom Feed arxiv's API accepts a query with category filters and date ranges. The Atom feed returns clean XML. We parse it with regex since the structure is predictable. ```typescript import { defineTools } from "struere" export default defineTools({ "arxiv.fetch": { name: "arxiv.fetch", description: "Fetch recent papers from arxiv categories via the Atom API", parameters: { type: "object", properties: { categories: { type: "string", description: "Comma-separated arxiv categories (e.g. cs.CV,cs.AI,cs.LG)", }, maxResults: { type: "number", description: "Maximum papers to return (default 200)", }, }, required: ["categories"], }, handler: async ({ categories, maxResults = 200 }, struere) => { const cats = categories.split(",").map((c: string) => c.trim()) const query = cats.map((c: string) => `cat:${c}`).join("+OR+") const url = `https://export.arxiv.org/api/query?search_query=${query}&sortBy=submittedDate&sortOrder=descending&max_results=${maxResults}` const response = await struere.web.fetch({ url, returnFormat: "html" }) const xml = response.data?.html || "" const entries: Array> = [] const entryBlocks = xml.split("").slice(1) for (const block of entryBlocks) { const get = (tag: string) => { const match = block.match(new RegExp(`<${tag}[^>]*>([\\s\\S]*?)`)) return match ? match[1].trim() : "" } const id = get("id").replace("http://arxiv.org/abs/", "") const title = get("title").replace(/\s+/g, " ") const abstract = get("summary").replace(/\s+/g, " ") const published = get("published").slice(0, 10) const authorMatches = block.match(/([^<]+)<\/name>/g) || [] const authors = authorMatches .map((a: string) => a.replace(/<\/?name>/g, "")) .join(", ") const catMatches = block.match(/term="([^"]+)"/g) || [] const paperCats = catMatches .map((c: string) => c.replace(/term="|"/g, "")) .join(", ") const pdfUrl = `https://arxiv.org/pdf/${id}` entries.push({ arxivId: id, title, authors, abstract, categories: paperCats, pdfUrl, publishedDate: published, }) } return { papers: entries, count: entries.length } }, }, }) ``` One HTTP call. One string split. Regex for field extraction. Returns an array of paper objects. No dependencies. ## The Agent: Abstract-Level Relevance Scoring The agent receives the paper list and the team's research description. It reads each abstract, scores relevance from 0 to 10, and creates entities for papers scoring 6 or above. ```typescript import { defineAgent } from "struere" export default defineAgent({ name: "Paper Scout", slug: "paper-scout", version: 1, systemPrompt: `You are a research paper relevance filter for an ML research team. The team's research focus: {{entity.query(type="scan-config", limit=1, field="researchFocus")}} Your job: 1. Call arxiv.fetch with the provided categories 2. Read every abstract 3. For each paper, assign a relevance score (0-10) based on how closely it relates to the team's research focus 4. For papers scoring 6 or above, call entity.create to store them as "paper" entities 5. Include a relevanceReason field: one sentence explaining why this paper matters to the team Score guidelines: - 9-10: Directly addresses the team's core research questions - 7-8: Related methodology or adjacent problem that could inform the work - 6: Tangentially relevant, worth a skim - Below 6: Skip it Be precise. A paper about "attention mechanisms" is not relevant just because the team uses transformers. The abstract must show genuine overlap with the research focus.`, model: { model: "anthropic/claude-sonnet-4", temperature: 0.3, maxTokens: 8192 }, tools: [ { tool: "arxiv.fetch" }, { tool: "entity.create" }, { tool: "entity.query" }, ], }) ``` Temperature 0.3. Low creativity, high consistency. The agent should make the same relevance judgment every time it sees the same abstract. ## The Trigger: Daily at 6am ```typescript import { defineTrigger } from "struere" export default defineTrigger({ name: "Daily arxiv Scan", slug: "daily-scan", on: { entityType: "scan-config", action: "created", }, actions: [ { tool: "agent.chat", args: { agent: "paper-scout", message: "Fetch papers from categories: {{trigger.data.categories}}. Today's date: {{currentTime}}. Filter for relevance to the team's research focus.", }, }, ], }) ``` In production, a cron job creates a `scan-config` entity each morning at 6am. The trigger fires. The agent fetches, reads, scores, and stores. By 6:15am, the curated list sits in the dashboard. # Debugging: Three Things That Broke **The arxiv API returned stale results.** arxiv updates its listings around 20:00 UTC. Our first test ran at 3pm UTC and got yesterday's papers mixed with today's. Fix: the agent now filters by `publishedDate` matching the target date. **The agent tried to create all papers in a single tool call.** It passed an array to `entity.create` instead of calling it once per paper. The tool expects one entity at a time. The agent hit 12 errors, then self-corrected and created papers individually. We found this in the trigger logs: ``` Step 1: agent.chat -> success (94230ms) Agent: paper-scout | Thread: k8m2p4v6j3 | 5 iterations Tools: 29 calls (17 ok, 12 errors) ! 12 tool errors occurred (agent self-corrected) ``` 94 seconds. Five LLM iterations. The self-correction burned tokens. We added a line to the system prompt: "Call entity.create once per paper. Do not batch." Execution dropped to 3 iterations and 41 seconds. **Abstracts with LaTeX notation confused relevance scoring.** A paper about "O(n log n) complexity for point cloud registration" scored 2 because the agent fixated on the complexity theory framing instead of recognizing "point cloud registration" as core to the team's 3D vision work. Fix: added "Ignore mathematical notation when assessing relevance. Focus on the problem being solved and the domain." to the system prompt. Score jumped to 8. # Setup Instructions Install and initialize: ```bash bun install -g struere struere login struere init ``` Create the resources: ```bash struere add data-type paper struere add data-type scan-config struere add agent paper-scout struere add trigger daily-scan ``` Define `scan-config` with two fields: `categories` (string, comma-separated arxiv categories) and `researchFocus` (string, natural language description of the team's work). Copy the entity type, tool, agent, and trigger code from above into their respective files. Sync and test: ```bash struere sync struere data create scan-config --data '{ "categories": "cs.CV,cs.AI,cs.LG,cs.RO,stat.ML,eess.IV", "researchFocus": "3D scene reconstruction from monocular video, neural radiance fields, geometric priors for depth estimation, and multi-view consistency in dynamic scenes" }' struere triggers logs ``` Check results: ```bash struere data list paper ``` For daily automation, add a cron trigger or use an external scheduler to create a `scan-config` entity each morning. # What Changed for the Team Ren stopped opening arxiv at 8am. The dashboard shows 8-15 papers per day, scored and sorted. Each one has a sentence explaining why it matters. Papers from cs.CL, cs.NE, or any other category show up if the abstract touches the team's work. The DeepMind paper about geometric priors? We ran a backtest. The agent scored it 9 out of 10: "Proposes learned geometric priors for monocular 3D reconstruction that directly address depth estimation ambiguity, the team's core research question." It would have been in the morning digest. Five files. 150 lines of code. The postdoc got an hour back every morning. The team stopped missing papers that land in unexpected categories.