Predicting Human Brain States with Transformer

The human brain is a complex and highly dynamic system, and our current knowledge of its functional mechanism is still very limited. Fortunately, with functional magnetic resonance imaging (fMRI), we can observe blood oxygen level-dependent (BOLD) changes, reflecting neural activity, to infer brain states and dynamics. In this paper, we ask the question of whether the brain states represented by the regional brain fMRI can be predicted. Due to the success of self-attention and the transformer architecture in sequential auto-regression problems (e.g., language modelling or music generation), we explore the possibility of the use of transformers to predict human brain resting states based on the large-scale high-quality fMRI data from the human connectome project (HCP). Current results have shown that our model can accurately predict the brain states up to 5.04 s with the previous 21.6 s. Furthermore, even though the prediction error accumulates for the prediction of a longer time period, the generated fMRI brain states reflect the architecture of functional connectome. These promising initial results demonstrate the possibility of developing generative models for fMRI data using self-attention that learns the functional organization of the human brain. Our code is available at: https://github.com/syf0122/brain_state_pred.