img

Language Reasoning Models can overtake LLMs...

Here's my quick 3 minute breakdown:

1. o1-preview: 97.8% on PlanBench Blocksworld vs. 62.5% for top LLMs, indicating shift from retrieval to reasoning.
2. 52.8% on obfuscated "Mystery Blocksworld" vs. near-zero for LLMs, suggesting abstract reasoning skills, showing transfer capability.
3. Variable "reasoning tokens" usage correlates with problem difficulty, hinting at internal search process, indicating adaptive compute.

The ability to plan a course of action that achieves a desired state of affairs has long been considered a core competence of intelligent ag...

https://arxiv.org/pdf/2409.13373v1

img
img

Isaiah Lee

Google

9 days ago

img

Kendall Gabriel

Stealth

9 days ago

img

Matilda Carmden

Advanced Micro Devices

8 days ago

img

Jordon Lee

Student

9 days ago

img

Coy Dean

Advanced Micro Devices

8 days ago

See more comments
img

Jordon Everett

Student

8 days ago

Sign in to a Grapevine account for the full experience.

Discover More

Curated from across

  • Home
  • Language Reasoning Models can overtake LLMs...