Figure 1
The diagram shows a process starting with MRI scans and clinical notes. These inputs are processed through a CNN-based feature extraction network, depicted as a series of layered rectangles. The extracted features are then fed into a transformer-based ChatGPT network, represented by a swirl icon. The output from the transformer network goes through a classification layer, illustrated as interconnected nodes. The final output includes tumor/non-tumor classification and natural language description, symbolized by a brain, a computer chip, and a report document.Model architecture