Thank you, Gwern. The post from today that you link is fascinating. It makes it obvious that the design I’m toying with is not radical enough. Fascinating to see how the web browsing model uses the internet as a sort of knowledge graph. I would have thought it too messy.
I wonder what Paul Graham would make of this. Judging from his essays on writing, he seems to be of the opinion that the process of putting your abstract ideas into words as best you can is a crucial part of the actual thinking — the part where you discover that maybe your arguments aren't as plausible as they were in your head or that you don't really believe what you thought you believed.
On the other hand, maybe that thinking-by-writing-part of the process can also be done by taking elaborate notes in a software like Obsidian (or, if you're an analog enthusiast like me, in a notebook).
Do you find yourself refining your thinking when trying to put your thoughts into sensible words, Henrik?
I'm pretty sure you could do repeated prompting + RL finetuning to expand outlines into books, the inverse of https://openai.com/blog/summarizing-books/ . Or consider today's post, finding a lot of snippets on the Internet and collating them and then writing a larger summary/argument based on them: https://openai.com/blog/improving-factual-accuracy/
Thank you, Gwern. The post from today that you link is fascinating. It makes it obvious that the design I’m toying with is not radical enough. Fascinating to see how the web browsing model uses the internet as a sort of knowledge graph. I would have thought it too messy.
I wonder what Paul Graham would make of this. Judging from his essays on writing, he seems to be of the opinion that the process of putting your abstract ideas into words as best you can is a crucial part of the actual thinking — the part where you discover that maybe your arguments aren't as plausible as they were in your head or that you don't really believe what you thought you believed.
On the other hand, maybe that thinking-by-writing-part of the process can also be done by taking elaborate notes in a software like Obsidian (or, if you're an analog enthusiast like me, in a notebook).
Do you find yourself refining your thinking when trying to put your thoughts into sensible words, Henrik?