cream cat

With the explosion of AI technology, especially the emergence of GPT, this word has been flashing in my mind according to OpenAI's explanation of their model's capabilities. They don't know why GPT has shown extraordinary performance since 3.5, or they can't explain this performance in principle. In fact, this word has long existed.

What is the emergence phenomenon? According to the professional definition, the emergence phenomenon refers to the unexpected overall properties or behaviors that appear in a system due to a large number of tiny interactions and interactions. These overall properties or behaviors cannot be simply understood or predicted by analyzing the behaviors of individual components in the system, but require consideration of the overall interaction of the system. A famous example is the behavior of ants. The behavior of a single ant is very simple, but when they interact with each other in a certain way, the entire ant colony can exhibit very complex behaviors, such as building ant nests, searching for food, and protecting the ant colony.

The same principle applies to large AI models. With the increase in training data, the originally simple predictive models suddenly acquire logical understanding capabilities. Because this ability can bring better predictions, better predictions continue to improve logical abilities, resulting in a flywheel effect and emergence. Another example is social networks. When people continue to connect and influence each other in social interactions, in turn, social networks themselves will also have an impact on individuals, and the interaction between each other leads to unexpected states or situations in the entire network.

Many domestic experts have always been skeptical about whether AI has logical abilities. They believe that language models cannot generate logic. The argument is also simple: logical abilities are only possessed by higher intelligent organisms, such as humans. However, humans do not have this ability at birth. When humans are born, they only have instinctive reactions, regardless of their breathing ability or feeding ability. The development of brain neurons matures as humans grow, and it is not considered fully developed until after the age of 20. So why is AI different?

In addition, AI has advantages over humans. Whether it is the number of cells or neurons, humans have theoretical limits and cannot grow and expand indefinitely. However, as an electronic computer based on silicon, its computing chips can be infinitely expanded and upgraded. The latest A100 from NVIDIA uses 7nm process technology, and TSMC will soon mass-produce process technology below 5nm, which means that future high-performance GPUs will have even more powerful computing capabilities than they currently do.

As for the future potential of GPT, there are different opinions in the industry, but according to the current iteration speed, I believe that within 2 years, generative models will be able to successfully complete video creation tasks. In addition, several companies in mainland China are also developing related products, such as Wenyanyixin. However, considering the long-standing restrictions on the cultural industry in China, how can large language models be reviewed? After all, hot topics are dynamic, and censorship is also dynamic. If the model cannot achieve self-censorship, it will not be promoted by the mainland government. Just look at Bitcoin and crypto.