cream cat

With the explosion of AI technology, especially the rise of GPT, this term has been flashing in my mind. According to OpenAI's explanation of their model's capabilities, they don't know why GPT has shown extraordinary performance since 3.5, or why they can't explain this performance in principle. In fact, this term has long existed.

What is the emergence phenomenon? According to the professional definition, the emergence phenomenon refers to the unexpected overall properties or behaviors that appear in a system due to a large number of tiny interactions and interactions. These overall properties or behaviors cannot be simply understood or predicted by analyzing the behavior of individual components in the system, but require considering the interaction of the entire system. A famous example is the behavior of ants. The behavior of a single ant is very simple, but when they interact with each other in a certain way, the entire ant colony can exhibit very complex behaviors, such as building ant nests, searching for food, and protecting the ant colony.

The same principle applies to large AI models. With the increase in training data, the originally simple predictive models suddenly acquire logical understanding capabilities. Because this ability can bring better predictions, better predictions further improve logical abilities, resulting in a flywheel effect and emergence. Another example is social networks. When people continue to connect and influence each other in social interactions, in turn, social networks themselves will also have an impact on individuals. The interaction between each other leads to unexpected states or situations in the entire network.

Many domestic experts have always been skeptical about whether AI has logical abilities. They believe that language models cannot generate logic. The argument is also simple: logical abilities are only possessed by higher intelligent organisms, such as humans. However, humans do not have this ability at birth. When humans are born, they only have instinctive reactions, whether it is breathing ability or feeding ability. The development of brain neurons matures gradually as humans grow, and it is not considered complete until after the age of 20. So why is AI different?

In addition, AI has advantages over humans. Whether it is the number of cells or neurons, humans have theoretical limits and cannot grow and expand indefinitely. However, as a silicon-based electronic computer, its computing chips can be infinitely expanded and upgraded. The latest A100 used by NVIDIA is based on 7nm process technology, and TSMC will soon mass-produce process technology below 5nm, which means that future high-performance GPUs will exert even more powerful computing capabilities than they currently do.

As for the potential of GPT in the future, opinions in the industry vary, but based on the current iteration speed, I believe that within 2 years, generative models will be able to successfully complete video creation tasks. In addition, several companies in mainland China are also developing related products, such as Wenyanyixin. However, considering the long-standing restrictions on the cultural industry in China, how can large language models be reviewed? After all, hot topics are dynamic, and censorship is also dynamic. If the model cannot achieve self-censorship, it will not be promoted by the mainland government. Just look at Bitcoin and crypto.