Speaker
Description
Aims: Visual system of mammals is comprised of two parallel pathways: the “what” and the “where” pathways which are specialized for object categorization and movement, respectively. An Artificial Neural Network (ANN) model of mammalian visual systems should also have the same specialized parallel pathways. Previously (Bakhtiari et al., NeurIPS, 2021), we showed that the “what” and “where” specializations emerged in an ANN with two parallel pathways trained on visual prediction. However, in this two-pathway model, the specialized role that each pathway acquires changes depending on the random seed used for initialization. How can we control the specialized role of each pathway prior to training? The factors that determine the final, specialized role of each pathway in an ANN are yet unclear. In this work, we studied the role of pathways’ initializations and path-specific additional loss functions in setting the emerging roles of “what” and “where” specializations in an ANN.
Methods: We trained ANNs with two segregated parallel pathways with a combination of path-specific and shared self-supervised loss functions. We quantified the specialization of the trained pathways with downstream tasks, such as motion direction discrimination. We examined the relationship between pathways’ learned motion specialization and the statistical properties of their random initializations.
Results: We showed that training an ANN that had two parallel pathways with a self-supervised prediction loss function was sufficient for learning both “what”- and “where”-like representations. Additional, path-specific loss functions were necessary to further encourage and stabilize the learning of each type of representation within the two pathways. For example, adding the self-motion estimation loss to one of the two pathways led to learning a better “where”-like pathway in our ANN. However, the additional path-specific losses could not fully control the final, specialized role that each pathway acquired through training. In fact, the final roles of the pathways were mainly determined at the initialization driven by the slight statistical asymmetries between the initializations of the two pathways. Creating an asymmetry via pretraining one of the pathways on self-motion estimation before joint predictive training of the two pathways was sufficient for controlling the specialized role of the pathways.
Conclusions: Our study suggests that an ANN with segregated parallel pathways trained with a combination of a global self-supervised prediction and path-specific loss functions can explain the development of parallel specialized pathways in mammalian visual systems. The statistical asymmetries between the two pathways mainly determine the role that each pathway acquires through training. Future work will examine whether these same properties develop in ANNs with macroscopic architectures that more closely resemble the human brain.
