Curiosity vector and learning
As someone who daily collects, analyzes, and evaluates various datasets for LLM training, while raising a young child in our household, I think a lot about the parallel between how LLMs learn and how children learn.
Pre-training is all about learning the relationship between different words, or more precisely, tokens. This for children happens from the second they are born. Even before they can see well, they apparently hear everything perfectly. They hear a certain word repeatedly said to them - their name - and learn that it is something related to themselves. They learn that the words mommy and daddy seem to have something to do with people who are always with them. They repeat words they hear, learn those words' association with certain people, objects around them, and particular circumstances, and their learning continues to expand their world. This is broadly what pretraining does.
Then they start to learn things in the format of input and output. This is where my daughter currently is. She is, right now, in front of me, learning how to write a Kanji, 田(ta). She is learning the relationship between the character(input), and the sound(ta), and the meaning(rice field). She is also learning basic math concepts such as addition, subtraction, multiplication, shapes, and so on. This is supervised fine-tuning. This phase lasts for a very long time throughout the school years.
Many people stop learning at the various points of this stage. Some will continue to learn in various domains. This phase of learning is not about getting to the right answer that was prepared for them. It is about having their own compass in their life direction that is set by a certain set of values, and continuing to move without a predetermined goal. This is reinforcement learning.
One aspect of all of this that is often missed in my opinion is what fuels all of this. With the case of LLMs, it's electricity. What is equivalent of electricity for people? Food is one thing that is needed for all of this but that is not sufficient. Some people continue learning through out their life whereas many others stop learning or limit their learning at some point in their life regardless of their level of education. Many of my physician friends seem to have stopped learning despite their massive intellectual capabilities. They make enough money - a lot of money in America - so why change anything?, they say. Their primary motivation that guide their life here is a good life and money needed for it. People who continue to learn are fueled by something completely different. Yes, some are driven by potential money opportunities but that's extremely rare if someone is already making half a million a year at their regular job.
It's curiosity.
Those who continue to learn never lose the sparkles in their eyes. They are driven primarily by their curiosity. They have almost madman-level of curiosity that drives them to keep going.
And what drives curiosity? Is it the value system they grew up with? Is it their genetics that determine their brain response to various stimulations? I am not exactly sure but I noticed that most people whose sparkles in their eyes never die show certain traits that are often seen as ASD traits. It seems to have something to do with their ability to sincerely not care about what the society expects of them. They just want to pursue whatever they want and not get dragged by everything around them. A sum of two vectors: one is driven by the curiosity and another exerted by external pressures. The magnitude of the resultant vector in the direction of curiosity determines whether, how long, and how far someone pursues RL in their life.
Japanese education is doing the opposite of this. There're plenty of curious individuals but the power of others in Japanese society is so strong that the curiosity vector ends up being very small. One good thing for them is that the society is very stable whereas many in America that is a big distraction.
For a curious mind to thrive, we need an environment that maximizes the curiosity vector.