lkaesberg/SPaRC
SPaRC, EMNLP 2025: 1,000 grid pathfinding puzzles that test spatial reasoning in LLMs. pip install sparc-puzzle to evaluate any model with the CLI or Inspect AI. Humans solve 98.0% of the test puzzles, the best model 83.6%.
GitHub repository with 6 stars and 3 forks.
Language: Python
Topics: benchmark, dataset, emnlp-2025, inspect-ai, large-language-models, llm, llm-evaluation, pathfinding, puzzles, reasoning