hermes-agent/skills/research/arxiv/SKILL.md
Teknium c49fa88b80
refactor(skills): shipped-set slim — 15 to optional, github 6-way merge, pdf absorbs OCR, channel-gated teams pipeline (index −26%) (#98539)
* refactor(skills): shipped-set slim — 15 skills to optional, github six-way merge, pdf absorbs OCR+nano-pdf, channel-gated teams pipeline

Maintainer-directed shipped-skills curation (skills index 1,900 -> ~1,400
tok/call on desktop; every session pays the index, so this is a per-call
diet on all installs):

- optional-skills moves (installable via skills hub, history preserved):
  creative comfyui/ascii-art/excalidraw/pretext/sketch/touchdesigner-mcp;
  ALL of mlops (huggingface-hub, llama-cpp, serving-llms-vllm,
  weights-and-biases, evaluating-llms-harness — subcategory structure
  kept); research-paper-writing (55 supporting files, 17.3K-tok load);
  openhue; blogwatcher (first taught the cronjob monitor-field watch
  pattern + web_extract instead of pre-cron manual workflows)
- DELETED session-librarian (Aug-12 'inspired by Perplexity Computer'
  port, never maintainer-intended; session_search covers discovery)
- github: six skills (auth, issues, pr-workflow, issue-to-pr,
  code-review, repo-management) merged into ONE software-development/
  github skill — routing body + complete per-workflow references;
  benbarclay authorship credited; codebase-inspection rides along;
  discipline pins from test_github_issue_to_pr_skill.py preserved
  against the reference body in the new test_github_skill.py
- pdf absorbs ocr-and-documents + nano-pdf as references/ + scripts
  (extract_pymupdf, extract_marker converted to the argparse house
  standard its contract test enforces)
- NEW session_platforms frontmatter gate (metadata.hermes): hides a
  skill from the index on gateway channels it is not for; fail-open on
  unknown platform; teams-meeting-pipeline gated to [teams, cron]
- blocked-page-recovery: research -> new web category; trigger-first
  description ('Use when a fetch fails: 403/429, paywall, WAF, bot
  wall.') so the model actually reaches for it on blocked fetches
- docs regenerated via generate-skill-docs.py (195 pages); related_skills
  swept repo-wide; tests: 1672 passed (2 openclaw failures pre-existing
  on clean main, Windows-local)

* chore: ignore .skills_prompt_snapshot.json (local index cache, accidentally committed)
2026-08-30 04:53:39 -07:00

9.8 KiB

name description version author license platforms metadata
arxiv Search arXiv papers by keyword, author, category, or ID. 1.0.0 Hermes Agent MIT
linux
macos
windows
hermes
tags related_skills
Research
Arxiv
Papers
Academic
Science
API
pdf

arXiv Research

Search and retrieve academic papers from arXiv via their free REST API. No API key, no dependencies — just curl.

Quick Reference

Action Command
Search papers curl "https://export.arxiv.org/api/query?search_query=all:QUERY&max_results=5"
Get specific paper curl "https://export.arxiv.org/api/query?id_list=2402.03300"
Read abstract (web) web_extract(urls=["https://arxiv.org/abs/2402.03300"])
Read full paper (PDF) web_extract(urls=["https://arxiv.org/pdf/2402.03300"])

Searching Papers

The API returns Atom XML. Parse with grep/sed or pipe through python for clean output.

curl -s "https://export.arxiv.org/api/query?search_query=all:GRPO+reinforcement+learning&max_results=5"

Clean output (parse XML to readable format)

curl -s "https://export.arxiv.org/api/query?search_query=all:GRPO+reinforcement+learning&max_results=5&sortBy=submittedDate&sortOrder=descending" | python -c "
import sys, xml.etree.ElementTree as ET
ns = {'a': 'http://www.w3.org/2005/Atom'}
root = ET.parse(sys.stdin).getroot()
for i, entry in enumerate(root.findall('a:entry', ns)):
    title = entry.find('a:title', ns).text.strip().replace('\n', ' ')
    arxiv_id = entry.find('a:id', ns).text.strip().split('/abs/')[-1]
    published = entry.find('a:published', ns).text[:10]
    authors = ', '.join(a.find('a:name', ns).text for a in entry.findall('a:author', ns))
    summary = entry.find('a:summary', ns).text.strip()[:200]
    cats = ', '.join(c.get('term') for c in entry.findall('a:category', ns))
    print(f'{i+1}. [{arxiv_id}] {title}')
    print(f'   Authors: {authors}')
    print(f'   Published: {published} | Categories: {cats}')
    print(f'   Abstract: {summary}...')
    print(f'   PDF: https://arxiv.org/pdf/{arxiv_id}')
    print()
"

Search Query Syntax

Prefix Searches Example
all: All fields all:transformer+attention
ti: Title ti:large+language+models
au: Author au:vaswani
abs: Abstract abs:reinforcement+learning
cat: Category cat:cs.AI
co: Comment co:accepted+NeurIPS

Boolean operators

# AND (default when using +)
search_query=all:transformer+attention

# OR
search_query=all:GPT+OR+all:BERT

# AND NOT
search_query=all:language+model+ANDNOT+all:vision

# Exact phrase
search_query=ti:"chain+of+thought"

# Combined
search_query=au:hinton+AND+cat:cs.LG

Sort and Pagination

Parameter Options
sortBy relevance, lastUpdatedDate, submittedDate
sortOrder ascending, descending
start Result offset (0-based)
max_results Number of results (default 10, max 30000)
# Latest 10 papers in cs.AI
curl -s "https://export.arxiv.org/api/query?search_query=cat:cs.AI&sortBy=submittedDate&sortOrder=descending&max_results=10"

Fetching Specific Papers

# By arXiv ID
curl -s "https://export.arxiv.org/api/query?id_list=2402.03300"

# Multiple papers
curl -s "https://export.arxiv.org/api/query?id_list=2402.03300,2401.12345,2403.00001"

BibTeX Generation

After fetching metadata for a paper, generate a BibTeX entry:

{% raw %}

curl -s "https://export.arxiv.org/api/query?id_list=1706.03762" | python -c "
import sys, xml.etree.ElementTree as ET
ns = {'a': 'http://www.w3.org/2005/Atom', 'arxiv': 'http://arxiv.org/schemas/atom'}
root = ET.parse(sys.stdin).getroot()
entry = root.find('a:entry', ns)
if entry is None: sys.exit('Paper not found')
title = entry.find('a:title', ns).text.strip().replace('\n', ' ')
authors = ' and '.join(a.find('a:name', ns).text for a in entry.findall('a:author', ns))
year = entry.find('a:published', ns).text[:4]
raw_id = entry.find('a:id', ns).text.strip().split('/abs/')[-1]
cat = entry.find('arxiv:primary_category', ns)
primary = cat.get('term') if cat is not None else 'cs.LG'
last_name = entry.find('a:author', ns).find('a:name', ns).text.split()[-1]
print(f'@article{{{last_name}{year}_{raw_id.replace(\".\", \"\")},')
print(f'  title     = {{{title}}},')
print(f'  author    = {{{authors}}},')
print(f'  year      = {{{year}}},')
print(f'  eprint    = {{{raw_id}}},')
print(f'  archivePrefix = {{arXiv}},')
print(f'  primaryClass  = {{{primary}}},')
print(f'  url       = {{https://arxiv.org/abs/{raw_id}}}')
print('}')
"

{% endraw %}

Reading Paper Content

After finding a paper, read it:

# Abstract page (fast, metadata + abstract)
web_extract(urls=["https://arxiv.org/abs/2402.03300"])

# Full paper (PDF → markdown via Firecrawl)
web_extract(urls=["https://arxiv.org/pdf/2402.03300"])

For local PDF processing, see the ocr-and-documents skill.

Common Categories

Category Field
cs.AI Artificial Intelligence
cs.CL Computation and Language (NLP)
cs.CV Computer Vision
cs.LG Machine Learning
cs.CR Cryptography and Security
stat.ML Machine Learning (Statistics)
math.OC Optimization and Control
physics.comp-ph Computational Physics

Full list: https://arxiv.org/category_taxonomy

Helper Script

The scripts/search_arxiv.py script handles XML parsing and provides clean output:

python scripts/search_arxiv.py "GRPO reinforcement learning"
python scripts/search_arxiv.py "transformer attention" --max 10 --sort date
python scripts/search_arxiv.py --author "Yann LeCun" --max 5
python scripts/search_arxiv.py --category cs.AI --sort date
python scripts/search_arxiv.py --id 2402.03300
python scripts/search_arxiv.py --id 2402.03300,2401.12345

No dependencies — uses only Python stdlib.


arXiv doesn't provide citation data or recommendations. Use the Semantic Scholar API for that — free, no key needed for basic use (1 req/sec), returns JSON.

Get paper details + citations

# By arXiv ID
curl -s "https://api.semanticscholar.org/graph/v1/paper/arXiv:2402.03300?fields=title,authors,citationCount,referenceCount,influentialCitationCount,year,abstract" | python -m json.tool

# By Semantic Scholar paper ID or DOI
curl -s "https://api.semanticscholar.org/graph/v1/paper/DOI:10.1234/example?fields=title,citationCount"

Get citations OF a paper (who cited it)

curl -s "https://api.semanticscholar.org/graph/v1/paper/arXiv:2402.03300/citations?fields=title,authors,year,citationCount&limit=10" | python -m json.tool

Get references FROM a paper (what it cites)

curl -s "https://api.semanticscholar.org/graph/v1/paper/arXiv:2402.03300/references?fields=title,authors,year,citationCount&limit=10" | python -m json.tool

Search papers (alternative to arXiv search, returns JSON)

curl -s "https://api.semanticscholar.org/graph/v1/paper/search?query=GRPO+reinforcement+learning&limit=5&fields=title,authors,year,citationCount,externalIds" | python -m json.tool

Get paper recommendations

curl -s -X POST "https://api.semanticscholar.org/recommendations/v1/papers/" \
  -H "Content-Type: application/json" \
  -d '{"positivePaperIds": ["arXiv:2402.03300"], "negativePaperIds": []}' | python -m json.tool

Author profile

curl -s "https://api.semanticscholar.org/graph/v1/author/search?query=Yann+LeCun&fields=name,hIndex,citationCount,paperCount" | python -m json.tool

Useful Semantic Scholar fields

title, authors, year, abstract, citationCount, referenceCount, influentialCitationCount, isOpenAccess, openAccessPdf, fieldsOfStudy, publicationVenue, externalIds (contains arXiv ID, DOI, etc.)


Complete Research Workflow

  1. Discover: python scripts/search_arxiv.py "your topic" --sort date --max 10
  2. Assess impact: curl -s "https://api.semanticscholar.org/graph/v1/paper/arXiv:ID?fields=citationCount,influentialCitationCount"
  3. Read abstract: web_extract(urls=["https://arxiv.org/abs/ID"])
  4. Read full paper: web_extract(urls=["https://arxiv.org/pdf/ID"])
  5. Find related work: curl -s "https://api.semanticscholar.org/graph/v1/paper/arXiv:ID/references?fields=title,citationCount&limit=20"
  6. Get recommendations: POST to Semantic Scholar recommendations endpoint
  7. Track authors: curl -s "https://api.semanticscholar.org/graph/v1/author/search?query=NAME"

Rate Limits

API Rate Auth
arXiv ~1 req / 3 seconds None needed
Semantic Scholar 1 req / second None (100/sec with API key)

Notes

  • arXiv returns Atom XML — use the helper script or parsing snippet for clean output
  • Semantic Scholar returns JSON — pipe through python -m json.tool for readability
  • arXiv IDs: old format (hep-th/0601001) vs new (2402.03300)
  • PDF: https://arxiv.org/pdf/{id} — Abstract: https://arxiv.org/abs/{id}
  • HTML (when available): https://arxiv.org/html/{id}
  • For local PDF processing, see the ocr-and-documents skill

ID Versioning

  • arxiv.org/abs/1706.03762 always resolves to the latest version
  • arxiv.org/abs/1706.03762v1 points to a specific immutable version
  • When generating citations, preserve the version suffix you actually read to prevent citation drift (a later version may substantially change content)
  • The API <id> field returns the versioned URL (e.g., http://arxiv.org/abs/1706.03762v7)

Withdrawn Papers

Papers can be withdrawn after submission. When this happens:

  • The <summary> field contains a withdrawal notice (look for "withdrawn" or "retracted")
  • Metadata fields may be incomplete
  • Always check the summary before treating a result as a valid paper