P-2A.38

Humans and 3D neural field models make similar 3D shape judgements

Thomas O'Connell, MIT, United States; Tyler Bonnen, Stanford University, United States; Yoni Friedman, Ayush Tewari, Josh Tenenbaum, Vincent Sitzmann, Nancy Kanwisher, MIT, United States

Session:
Posters 2A Poster

Track:
Cognitive science

Location:
North Schools

Presentation Time:
Fri, 25 Aug, 13:00 - 15:00 United Kingdom Time

Abstract:
Human visual perception captures the 3D shape of objects. While convolutional neural networks (CNNs) resemble some aspects of human visual processing, they fail to explain human shape perception. A new deep learning approach, 3D neural fields (3D-NFs), has driven remarkable recent progress in 3D graphics and computer vision. 3D-NFs encode the geometry of objects in a continuous, coordinate-based representation. Here, we investigate whether humans and 3D-NFs make similar trial-level 3D shape judgments on match-to-sample tasks with rendered stimuli. In Experiment 1, 3D-NF behavior is more similar to human behavior than standard CNNs trained on ImageNet, regardless of whether lure objects were a.) from a different category than the target, b.) the same category as the target, or c.) matched to have the most similar 3D-NF to the target as possible. In Experiment 2, to accentuate differences between humans and 3D-NFs compared to CNNs, five difficulty conditions were defined based on the performance of 25 ImageNet CNNs. Again, we find 3D-NF and human behavior are well aligned, with both showing high accuracy even for trials where CNNs fail. Overall, 3D-NFs and humans show similar patterns of 3D shape judgements, suggesting 3D-NFs as a promising framework for investigating human 3D shape perception.

Manuscript:
License:
Creative Commons License
This work is licensed under a Creative Commons Attribution 3.0 Unported License.
DOI:
10.32470/CCN.2023.1603-0
Publication:
2023 Conference on Cognitive Computational Neuroscience
Presentation
Discussion
Resources
No resources available.
Session P-2A
P-2A.1: Meta-cognitive planning for learning representations
Tingke Shen, Peter Dayan, Mihály Bányai, Max Planck Institute for Biological Cybernetics, Germany
P-2A.2: Learning how to compute confidence
Pierre Le Denmat, KU Leuven, Belgium; Tom Verguts, Ghent University, Belgium; Kobe Desender, KU Leuven, Belgium
P-2A.3: Increased Dynamic Range as a Driver of Neural and Behavioral Differences in ASD
Oded Wertheimer, Yuval Hart, Hebrew university of jerusalem, Israel
P-2A.4: Dis-confirmatory evidence drives confidence
Annika Boldt, University College London, United Kingdom; Kobe Desender, Katholieke Universiteit Leuven, Belgium
P-2A.5: Inter-individual neural code conversion without paired stimuli
Haibao Wang, Jun Kai Ho, Fan Cheng, Shuntaro C. Aoki, Yukiyasu Kamitani, Kyoto University, Japan
P-2A.6: Reconstructing seen images from human brain activity via guided stochastic search
Reese Kneeland, Jordyn Ojeda, Ghislain St-Yves, Thomas Naselaris, University of Minnesota, United States
P-2A.7: Overcoming sensory-memory interference: three solutions and their experimental signatures
Andrii Zahorodnii, Christopher J. Cueva, Massachusetts Institute of Technology, United States
P-2A.8: Retrospective value bias in setting temporally extended goals in humans
Sneha Aenugu, John O'Doherty, California Institute of Technology, United States
P-2A.9: Binding via Combining Spike Synchrony and Generative Top-down Attention
Hao Zheng, Hui Lin, Center for Brain-Inspired Computing Research (CBICR), China; Sen Song, Tsinghua Laboratory of Brain and Intelligence, China; Rong Zhao, Center for Brain-Inspired Computing Research (CBICR), China
P-2A.10: Faster learning from slow features: The temporal coherence prior in human reinforcement learning
Noa Hedrich, Sam Hall-McMaster, Max Planck Institute for Human Development, Germany; Eric Schulz, Max Planck Institute for Biological Cybernetics, Germany; Nicolas Schuck, Max Planck Institute for Human Development, Germany
P-2A.11: Homeostatic Plasticity Enhances Robustness in Spiking Neural Networks
Mingkun Xu, Jianping Xiong, Jing Pei, Lei Deng, Tsinghua University, China
P-2A.12: Intuitive Physics in Text-Conditional Image Generation
Eli Kramer, Sarah Schwettmann, Pawan Sinha, MIT, United States
P-2A.13: Forgone, but not forgotten: separate episodic memories underlie explicit reports and eye movements
Yul Kang, Queen Mary University of London, United Kingdom; Johannes Mahr, Harvard University, United States; Márton Nagy, Krisztina Andrási, Eötvös Loránd University, Hungary; Gergely Csibra, Central European University, Austria; Máté Lengyel, University of Cambridge, United Kingdom
P-2A.14: 3D View Prediction Models of the Dorsal Visual Stream
Gabriel Sarch, Carnegie Mellon University, United States; Hsiao-Yu Fish Tung, Massachusetts Institute of Technology, United States; Aria Wang, Carnegie Mellon University, United States; Jacob Prince, Harvard University, United States; Michael Tarr, Carnegie Mellon University, United States
P-2A.15: Synergistic information supports modality integration and flexible learning in neural networks solving multiple tasks
Alexandra Proca, Fernando Rosas, Imperial College London, United Kingdom; Andrea Luppi, University of Cambridge, United Kingdom; Daniel Bor, Queen Mary University of London, United Kingdom; Matthew Crosby, Pedro Mediano, Imperial College London, United Kingdom
P-2A.16: Memory-based predictions prime perceptual judgements of upcoming scene views in immersive, real-world scenes
Anna Mynick, Thomas L. Botch, Allie Burrows, Adithi Jayaraman, Adam Steel, Caroline E. Robertson, Dartmouth College, United States
P-2A.17: Face-Off: Pitting Computational Models of the Fusiform Face Area Against Each Other with Controversial Face Stimuli
Wenxuan Guo, Columbia University, United States; Tal Golan, Ben-Gurion University of the Negev, Israel; Heiko Schütt, Nikolaus Kriegeskorte, Columbia University, United States
P-2A.18: On the Relative Stability of Deep Distributional Reinforcement Learning
Rex Liu, Bowen He, Michael Frank, Brown University, United States
P-2A.19: Humans use Newtonian physics in intuitive sensorimotor decisions under risk
Fabian Tatai, Dominik Straub, Constantin Rothkopf, Technichal University of Darmstadt, Germany
P-2A.20: A new take on model-based and model-free influences on mental effort and striatal prediction errors
Carolina Feher da Silva, University of Surrey, United Kingdom; Gaia Lombardi, Micah Edelson, Todd Hare, University of Zurich, Switzerland
P-2A.21: Computational drivers of advice-giving
Hernan Anllo, Department of cognitive studies, France; Gil Salamander, Department of cognitive sciences, Israel; Stefano Palminteri, Department of cognitive studies, France; Nichola Raihani, Department of experimental psychology, United Kingdom; Uri Hertz, Department of cognitive sciences, Israel
P-2A.22: Precision Brain Encoding Under Naturalistic Conditions
Dora Gozukara, Djamari Oetringer, Linda Geerligs, Umut Güçlü, Radboud University, Netherlands
P-2A.23: Effectiveness of different computational models during the Tacit Communication Game
Yuanwei Yao, Tatia Buidze, Jan Gläscher, University Medical Center Hamburg-Eppendorf, Germany
P-2A.24: Predictions of Specific Stimulus Features Bias Perceptual Decisions in the Brain
Yuening Yan, University of Glasgow, United Kingdom; Jiayu Zhan, Peking University, China; Oliver Garrod, Xuan Cui, Robin A.A. Ince, Philippe G. Schyns, University of Glasgow, United Kingdom
P-2A.25: Real-time virtual physical deformations: a continuous design paradigm for interactive reasoning
Sara Monteiro, Daniel McNamee, Champalimaud Foundation, Portugal
P-2A.26: Theta Phase Separates Object Features in Human Hippocampus during an Associative Memory Task
Marije ter Wal, University of Birmingham, United Kingdom; Juan Linde Domingo, Max Planck Institute for Human Development, Germany; Frederic Roux, University of Birmingham, United Kingdom; Luca Kolibius, University of Glasgow, United Kingdom; Stephanie Gollwitzer, Johannes Lang, Katrin Walther, Hajo Hamer, Universitätsklinikum Erlangen, Germany; David Rollings, Vijay Sawlani, Ramesh Chelvarajah, Queen Elizabeth Hospital Birmingham, United Kingdom; Bernhard Staresina, University of Oxford, United Kingdom; Simon Hanslmayr, Maria Wimber, University of Glasgow, United Kingdom
P-2A.27: Connectivity of different occipital body areas and their roles in affective action perception
Baichen Li, Marta Poyo Solanas, Giuseppe Marrazzo, Beatrice de Gelder, Maastricht University, Netherlands
P-2A.28: Deep learning-based decoding of spatial information from limbic-cortical local field potentials reveal drifting spatial representations with increasing stability
Kipp Freud, Matt Jones, Nathan Lepora, University of Bristol, United Kingdom; Cian O'Donnell, Ulster University, United Kingdom
P-2A.29: Statistical learning principles yield generalization and naturalistic behaviors in transitive inference
Samuel Lippl, Kenneth Kay, Columbia University, United States; Greg Jensen, Reed College, United States; Vincent P. Ferrera, L.F. Abbott, Columbia University, United States
P-2A.30: The Role of Protracted Accumulation in Prolonged Perceptual Decisions
Hadiseh Hajimohammadi, Kieran Mohr, Simon Kelly, University College Dublin, Ireland
P-2A.31: Neurocomputational Underpinnings of Cooperative Behavior
Arkady Konovalov, University of Birmingham, United Kingdom; Claire Lugrin, Christian C. Ruff, University of Zurich, Switzerland
P-2A.32: Increased neural encoding of reward expectation underlies impulsive choices
Rhiannon Cowan, Tyler Davis, University of Utah, United States; Bornali Kundu, UCSF, United States; John Rolston, Shervin Rahimpour, Harvard University, United States; Elliot Smith, University of Utah, United States
P-2A.33: History-Dependent Decision-Making across Species
Anne Urai, Leiden University, Netherlands; The International Brain Laboratory, The International Brain Laboratory, United States
P-2A.34: Helping and reciprocating in a sequential decision making task
Pamela Osborn Popp, Todd Gureckis, New York University, United States
P-2A.35: The Impact of Scene Context on Visual Object Recognition: Comparing Humans, Monkeys, and Computational Models
Sara Djambazovska, Swiss Federal Institute of Technology (EPFL), Switzerland; Anaa Zafer, Hamidreza Ramezanpour, York University, Canada; Gabriel Kreiman, Harvard Medical School, United States; Kohitij Kar, York University, Canada
P-2A.36: Isolating signatures of motor-independent evidence accumulation in monkey EEG
Katerina Kalou, Redmond O’Connell, Trinity College, Ireland; Simon Kelly, University College Dublin, Ireland; Tobias Teichert, University of Pittsburgh, United States
P-2A.37: A Rational Analysis of the Optimism Bias using Meta-Reinforcement Learning
Johannes A. Schubert, Akshay K. Jagadish, Marcel Binz, Eric Schulz, Max Planck Institute for Biological Cybernetics, Germany
P-2A.38: Humans and 3D neural field models make similar 3D shape judgements
Thomas O'Connell, MIT, United States; Tyler Bonnen, Stanford University, United States; Yoni Friedman, Ayush Tewari, Josh Tenenbaum, Vincent Sitzmann, Nancy Kanwisher, MIT, United States
P-2A.39: The Unreasonable Effectiveness of Word Models in Predicting High-Level Visual Cortex Responses to Natural Images
Colin Conwell, Jacob Prince, George Alvarez, Talia Konkle, Harvard University, United States
P-2A.40: Emergence of neuronal ensembles in a chaotic corticostriatal circuit
Alessia Cavallo, Interventional and Cognitive Neuromodulation, Movement Disorder and Neuromodulation Unit, Charité, Germany; Andrea Mattera, Giovanni Granato, Gianluca Baldassarre, Laboratory of Embodied Natural and Artificial Intelligence, Institute of Cognitive Sciences and Technologies, National Research Council, Italy
P-2A.41: Efficient Continual Learning in Reservoir Networks
Paul Okeahalam, Liang Zhou, Jorge Menendez, Peter Latham, University College London, United Kingdom
P-2A.42: Tasks collapse the intrinsic dimensionality of activity in non-selective cortex
Erica Busch, Tristan Yates, Nicholas Turk-Browne, Yale University, United States
P-2A.43: Two neural mechanisms of geometric shape perception in humans
Mathias Sablé-Meyer, University College London, United Kingdom; Lucas Benjamin, Fosca Al Roumi, Université Paris Saclay, INSERM, CEA, France; Stanislas Dehaene, Collège de France, France
P-2A.44: Sequential Learning and Retrieval in a Sparse Distributed Memory: The K-winner Modern Hopfield Network
Shaunak Bhandarkar, James McClelland, Stanford University, United States
P-2A.45: The structure of experience: Examining the emergence of schematic representations in the medial prefrontal cortex
Philipp C. Paulus, University of Freiburg, Germany; Angharad N. Williams, Nottingham Trent University, United Kingdom; Sina S. Wiese, Roland G. Benoit, Max Planck Institute for Human Cognitive and Brain Sciences, Germany
P-2A.46: Neural Network Modules Learn Specialized Functions Under Strong Resource Constraints
Gabriel Béna, Dan F. M Goodman, Imperial College London, United Kingdom
P-2A.47: Back to the present: self-supervised learning across cortical layers
Kevin Kermani Nejad, University of Bristol, United Kingdom; Loreen Hertäg, Technische Universität Berlin, Germany; Paul Anastasiades, Rui Ponte Costa, University of Bristol, United Kingdom
P-2A.48: Space as a Scaffold for Temporal Generalisation
Jacques Pesnot Lerousseau, Christopher Summerfield, University of Oxford, United Kingdom
P-2A.49: How do humans learn concepts and strategies?
Tsvetomira Dumbalska, Univeristy of Oxford, United Kingdom; Areej Bhatti, Queen Mary University, United Kingdom; Ibi Ali, University of Edinburgh, United Kingdom; Christopher Summerfield, University of Oxford, United Kingdom
P-2A.50: Grid-like coding of an abstract value space for prospective decision making
Alexander Nitsch, Mona M. Garvert, Jacob L. S. Bellmund, Max Planck Institute for Human Cognitive and Brain Sciences, Germany; Nicolas W. Schuck, Max Planck Research Group NeuroCode, Max Planck Institute for Human Development, Germany; Christian F. Doeller, Max Planck Institute for Human Cognitive and Brain Sciences, Germany
P-2A.51: Numerosity representation across changes in object and scene content in convolutional neural networks
Thomas Chapalain, Paris-Saclay University, France; Bertrand Thirion, Inria, France; Evelyn Eger, CEA, France
P-2A.52: Singular violations of transitivity disrupt inferred relational knowledge in humans and reinforcement learning models
Thomas Graham, Max Planck School of Cognition, Germany; Bernhard Spitzer, Max Planck Institute for Human Development, Germany
P-2A.53: The role of gaze for value encoding and recollection in orbitofrontal cortex
Demetrio Ferro, Anna Rifé Mata, Universitat Pompeu Fabra (UPF), Spain; Tyler Cash-Padgett, Maya Zhe-Wang, Benjamin Hayden, University of Minnesota (UMN), United States; Rubén Moreno Bote, Universitat Pompeu Fabra (UPF), Spain
P-2A.54: Causal Influence Decomposition of the Human Brain Using Extensive In-silico Perturbation Analysis
Kayson Fakhar, Fatemeh Hadaeghi, Shrey Dixit, Arnaud Messé, University Medical Center Hamburg-Eppendorf, Germany; Bratislav Misic, Montreal Neurological Institute and Hospital, Canada; Claus Christian Hilgetag, University Medical Center Hamburg-Eppendorf, Germany
P-2A.55: A controller-peripheral architecture and costly energy principle for learning
Xiaoliang Luo, Brett Roads, University College London, United Kingdom; Robert Mok, University of Cambridge, United Kingdom; Bradley Love, University College London, United Kingdom
P-2A.56: When to choose: Information seeking in the speed-accuracy tradeoff
Javier Masís, Princeton University, United States; David Melnikoff, Lisa Feldman Barrett, Northeastern University, United States; Jonathan Cohen, Princeton University, United States
P-2A.57: Curriculum learning for cognitive map formation for human navigation
Leonie Glitz, Helen C. Barron, Christopher Summerfield, University of Oxford, United Kingdom
P-2A.58: Probing Next-Word and Long-Distance Prediction Using Encoding Modelling and MEG
Inés Schönmann, Floris P. de Lange, Micha Heilbron, Donders Institute for Brain, Cognition and Behaviour, Radboud University, Netherlands
P-2A.59: EEG adaptation pattern reveals slow update of temporal predictions in autism
Yarden Weiss, Nathaniel Zuk, Merav Ahissar, The Hebrew University, Israel
P-2A.60: Neurocomputational characterisation of differences in multisensory processing in Autism and Schizophrenia
Amirreza Nadimi Shahraki, University of Leeds, United Kingdom; Maida Toumaian, Neurosciences and Precision Medicine Research Institute “COSTAS STEFANIS”, Greece; Jian Liu, University of Leeds, United Kingdom; Nikolaos Smyrnis, University General Hospital “ATTIKON”, United Kingdom; Ioannis Delis, University of Leeds, United Kingdom