Distinguished Professor
Director of the Center for Philosophy of Science
Center for Philosophy of Science
University of Pittsburgh
Abstract: In this talk, I will criticize neo-empiricism, i.e., the view that concepts and percepts are the same kind of representation, and I will show that the brain is an amodal system: Cognitive neuroscience supports Descartes against Hume. I also offer a novel interpretation of the body of evidence that was taken in the 1990s and 2000s to support neo-empiricist theories of concepts: the offloading hypothesis.
Very clear video and paper, thanks for contributing this, Edouard. As someone with generally empiricist tendencies but who has found your work on this topic persuasive, it seems to me the most interesting question going forward for the concept empiricist is whether there’s sense to be made of multi-modal perceptual representation, and what arguments might be offered in favor of this sort of take vs. an amodal take on the kinds of representations you discuss here (being open to the possibility that the remaining debate would be merely terminological). I have always thought that an empiricist view that did not allow for distinctively multi-modal representations was implausible, but it remains difficult for the concept empiricist to articulate exactly how perceptual format could become distinctively multi-modal as processing proceeds from the earlier to later cortical areas. I am now optimistic that some of work on hierarchical perceptual abstraction in deep neural networks might help us understand this; though if the networks are able to abstract some format that could be shared between different perceptual modalities of say the same number of objects, I guess I’m at a loss to say whether that resulting abstracted perceptual format should be called perceptual or amodal. One wouldn’t, I suppose, expect it to register on the various operational criteria you propose for perceptual format, but supposing that the neural network models are faithful in a relevant sense, one at the same time has a story about how that abstract format could be learned from perceptual experience–and moreover how one would get back from that abstracted format to modality-specific representations of exemplars for simulation and reasoning. Meaningfully empiricist perhaps in one sense with historical pedigree, if not clearly the sort of concept empiricism about format that you target?
The notion of multimodal representation is clearly part of the toolkit empiricists can appeal to in order to account for some of the evidence I discuss in the talk and paper. This is particularly clear for number and time representations, of the sort found in the IPS. If I remember correctly, Prinz appeals to it in his book on concepts.
An issue is that it adds yet another degree of freedom in the debate between the empiricists and the rationalists, one that is as ill defined as the notion of format and perceptual format. The way to go ahead then is to do what I suggest should be done, and to some extent is already done, with the notion of a perceptual format: identify plausible marks that suggest, although don’t define, the involvement of perceptual representations (e.g., transfer costs). The same thing then should be done for multimodal representation vs. amodal representation.
neural network models are tricky, and it is not clear to me whether the representations that get to be created count as amodal or perceptual or multimodal. How would you answer the question? the usual markers in psychology or neuroscience don’t apply there, and I am curious to know what kind of markers you are envisaging to replace them in this context.
Hi Edouard,
Thanks for this really interesting and concise paper.
I have a question about how you set up the distinction between neo-empiricist views of concepts and amodal views. You write that according to amodal views, “the format of concepts differs from the format or formats of perceptual and motor representations.” But one can hold that concepts are couched in an amodal language of thought (for example) while also holding that perceptual representations are couched in that same amodal language of thought. One could think that every mental representation, in perception as well as cognition, is couched in the same amodal format – I think this is Pylyshyn’s view, for example. Or you could endorse a pluralist view that perception delivers representations in proprietary formats as well as representations in an amodal format (I argue for this, and I think Fodor does as well). The difference between perception and cognition on such views might be informational encapsulation, or something else.
Do you agree that these positions are coherent? If so, then shouldn’t there be another way of setting up the difference between neo-empiricist and amodal views of concepts? It seems like the issue is really about whether the format of thought is proprietary to perception, rather than merely whether it’s shared with perception. There could be different ways of developing this characterization.
One way would be to give a specific account of the kinds of formats that are proprietary to specific perceptual modalities (e.g., that they’re iconic or analog), which you discuss in the paper. You could then set up the debate as being about whether all concepts are couched in those specific formats.
Another way would be to focus primarily on acquisition. I think the reason Jesse Prinz frames neo-empiricism as saying the formats of concepts are the same as whatever formats are used in perception is that he really takes the essence of neo-empiricism to be that concepts come from representations used in perception, which is denied by amodal theorists like Fodor and Pylyshyn. The fact that on Prinz’s view concepts are acquired from perception rather than vice versa might be enough to ground the claim that the format of concepts is proprietarily perceptual.
None of this is an objection to the thesis of your paper – it’s really just a concern about how we get an initial grip on the difference between neo-empiricist and amodal views.
Jake
Thanks for these interesting points.
I agree that the two positions you have sketched — Pylyshyn’s and Fodor’s — are coherent, and I agree that the coherence of Pylyshyn’s position is a challenge for my way of setting up the terms of the debate. (Incidentally, Prinz comments about this question in Furnishing the Mind.)
I will note however that it is a little bit unclear what Pylyshyn’s position exactly is. When Pylyshyn argues that all representations are linguistic, it is unclear whether he holds that there is a single language for all representations instead of several linguistic codes. I have a hard time taking the former hypothesis seriously since we do not seem able to form thoughts that involve both the primitives of early vision and high level concepts. But if the former were right, then Pylyshyn would then perhaps be a radical empiricist, just one of an unusual stripe.
On the other hand, I am not quite sure to see what you mean when you write, “the issue is really about whether the format of thought is proprietary to perception, rather than merely whether it’s shared with perception”.
And I would like to resist your reformulation of the debate: It is not fruitfully put in terms of digital vs. analog representations since we want to leave open the possibility that amodal representations are analog. Nor is it fruitfully cast in acquisition terms. An empiricist about format could be a nativist.
Rather, the relation between perceptual and conceptual representations (attentional selection vs. translation) and the nature of processing (does thinking involve perceptual reenactment) seems enough to distinguish the competing views at hand.
I agree with your last point that the process-based criteria you use to empirically pull the two hypotheses apart is a useful way of getting empirical traction on the distinction independently of the issues I raised.
However we define neo-empiricist vs. amodal theories of concepts, I think it shouldn’t turn out that Fodor and Pylyshyn are neo-empiricists. A lack of ability to compose early sensory representations with abstract concepts could be explained by appeal to encapsulation; that is, the representations could be in the same amodal format but be architecturally isolated from one another so they’re unable to compose together. (Though I think there is in fact evidence both low-level perceptual representations and more high-level concepts like PIANO compose together in object files.) One might hold that representations in the language faculty, for example, have the same amodal, language-like format as concepts, but they’re stuck in the language faculty and therefore can’t compose with other concepts. This kind of view applied to perception doesn’t seem to result in empiricism, since one could hold that there’s nothing about any of these representations that marks them as specifically sensory. At the very least this position is conceivable, so it can’t simply follow from the fact that one thinks perception and cognition use the same format that one is an empiricist.
The idea that there might be something about a representation that marks it as specifically sensory is what I meant to be getting at by distinguishing the view that concepts have a proprietarily perceptual format from the view that concepts and percepts have the same format. And what might mark a representation’s format as proprietarily perceptual could be its specific structure or its origin (rather than merely its being mentally ubiquitous).
On specific structure: The way I put the point about iconic/analog format was too coarse-grained – I didn’t mean that merely being analog or iconic would make a representation proprietarily perceptual. But there might be particular kinds of analog or iconic formats (or other formats entirely) that mark perceptual states. For example, the analog magnitude system and early color representations in vision might both be analog or iconic, but there might be more fine-grained syntactic differences between them (e.g., perhaps numerical analog magnitudes can’t compose with other features or spatiotemporal values the way iconic representations in perception can). This is all much too sketchy, but perhaps there could be a substantive way of characterizing the distinct formats used in perceptual systems – and the burden would be on the empiricist to show what those formats are beyond coarse-grained claims about analog vs. digital formats, and then to demonstrate that concepts reduce to them.
On origin: I agree that someone who thought that concepts represented in a proprietarily perceptual format could also accept that all concepts are innate and aren’t acquired by copying perceptual representations. I’m not quite sure why that view should be considered an empiricist one (as opposed to imagist, or something else) – I think it’s compatible with just about everything Chomsky has written, for example. And without the addition “distinctly sensory format,” we’re just left with an arch-rationalist view like Pylyshyn’s. But in any case, relying on acquisition would at least provide a substantive way of saying precisely why concepts are, for an empiricist, couched in a proprietarily perceptual format. Since concepts are (on the classical empiricist view endorsed by Prinz) just stored copies of perceptual states, it follows that the format of concepts is proprietary to perception. And that view would rule out Fodor and Pylyshyn, which seems like the right result.
Anyway, these are just puzzles about how to articulate exactly what empiricism is, and you’re right that one can argue against the empiricist as you do without settling these issues.