Testing LLM reasoning abilities with SAT is not an original idea; there is a recent research that did a thorough testing with models such as GPT-4o and found that for hard enough problems, every model degrades to random guessing. But I couldn't find any research that used newer models like I used. It would be nice to see a more thorough testing done again with newer models.
Microsoft says Copilot was summarizing confidential emails without permission，更多细节参见51吃瓜
Thanks for reading Brian's Substack! Subscribe for free to receive new posts and support my work.，推荐阅读体育直播获取更多信息
default = ["full"]
Number (8): Everything in this space must add up to 8. The answer is 3-3, placed vertically; 5-0, placed horizontally.