The third neuronal network was a relatively simple and processed language using vectorized representations of those “moving red” sentences. Finally, the fourth neuronal network functioned as an associative layer and predicted the production of the previous three in each time step. “When we do an action, we don’t always have to verbalize it, but we have this verbalization in our minds at some point,” says Vijayaraghavan. The AI that he and his team built was destined to do exactly that: connecting without problems the language, the proprioception, the planning of the action and the vision.
When the robotic brain was in operation, they began to teach some of the possible combinations of command and movements sequences. But they were not taught them all.
The birth of compositionality
In 2016, Breen Lake, professor of Psychology and Data Science, published a paper In which his team named a set of competitions, machines must really dominate to learn and think as humans. One of them was the composition: the ability to compose or decompose a whole in parts that can be reused. This reuse allows them to generalize the knowledge acquired to new tasks and situations. “The composition phase is when children learn to combine words to explain things. They [initially] Learn the names of objects, the names of the actions, but those are just words. When they learn this concept of composition, their ability to communicate explodes, ”explains Vijayaraghavan.
The AI that his team built was made for this exact purpose: see if he would develop compositionality. And he did.
Once the robot learned how certain commands and actions were connected, he also learned to generalize that knowledge to execute the commands that he had never heard before. Recognizing the names of the actions he had not performed and then made them in block combinations he had never seen. Vijayaraghavan’s AI discovered the concept of moving something to the right or left or putting an element on top of something. It could also combine words to name previously invisible actions, such as putting a blue block in a red one.