PolyU-led research reveals that sensory and motor inputs help large language models represent complex concepts (IMAGE)
Caption
a–c, The results for the individual-level pairwise correlation analysis for each dimension in the non-sensorimotor (a), sensory (b) and motor (c) domains. This analysis aims to examine the similarity between human and model conceptual representations while considering individual variability. The x axis represents the Spearman correlation coefficient, while the y axis shows the kernel density estimation of the correlation distributions. Notably, in the subplots for the motor dimensions, the y axis displays higher density peaks due to PaLM yielding model–human similarities clustered around zero. Cohen’s d is reported for each dimension to quantify the standardized distance between the human–human and model–human correlation distributions. The d values for GPT-3.5, GPT-4, PaLM and Gemini models are presented between forward slashes (‘/’), respectively. A negative d value, highlighted in purple, indicates that the model–human similarities are greater than the human–human similarities for that particular dimension and model. The distribution curves for human–human pairwise similarity, serving as benchmarks, are visually distinguished by the increased line thickness. When the colours are filled in model–human similarity distribution curves, they indicate that there is no credible evidence those model–human similarities are lower than human–human similarities (non-sensorimotor: 16 distributions out of 28 model–human distributions,16/12; sensory: 4/20; motor: 2/18). These filled-in curves highlight the dimensions and models where the model-generated ratings align closely with human ratings at the individual level. Here, the P values were assessed with two-sided t-tests and corrected for multiple comparisons using the FDR.
Credit
© 2025 Research and Innovation Office, The Hong Kong Polytechnic University. All Rights Reserved.
Usage Restrictions
nil
License
Original content