Retinal images and object files: towards empirically evaluating philosophical accounts of visual perspective

23 thoughts on “Retinal images and object files: towards empirically evaluating philosophical accounts of visual perspective”

1. Introduction

   In his interesting paper “Retinal images and object files: towards empirically evaluating philosophical accounts of visual perspective,” Assaf investigates whether it is possible to adduce empirical evidence for or against the Relational Property View (RPV). We can express RPV as the conjunction of two claims: (i) RPV explains the perspectival character of perceptual experience, say, for example, the elliptical appearance of a tilted coin, by appeal to representations of perspectival properties, and (ii) RPV construes perspectival properties as relational properties of external objects. In his paper, Assaf proceeds roughly as follows. He first distinguishes RPV from an alternative view, namely the view that perspectival properties are intrinsic properties of the retina (RET). He then argues that RPV is not in conflict with vision science. Finally, and most importantly, he discusses possible experiments that would allow us to decide between RPV and RET. These experiments are based on the influential object files paradigm.

   Assaf’s paper is a welcome addition to the existing literature on this topic. As Assaf points out, RPV has been evaluated mainly as a philosophical claim­—arguments for and against it have been based on epistemological, phenomenological, and other purely philosophical grounds. It is therefore important to also consider the empirical plausibility of RPV.

   My commentary is organized in two sections. In the first section, I present some preliminary considerations about RPV. The aim of these considerations is to clarify RPV in two respects that, I hope, are in line with Assaf’s characterization of this view. In the second section, I consider some of Assaf’s proposed experiments. The purpose of this section is to formulate a number of questions that might help to further refine these experiments.

    

   1. Preliminary considerations: RPV and RET

   RPV aims at an explanation of the perspectival character of perceptual experience. We can illustrate the perspectival character of perceptual experience with the example of a round coin. Let’s assume that the conditions of observation are normal and that you do not misperceive the coin’s shape. When you look at the coin from a perpendicular angle, you will see it as being round, that is, you will see its intrinsic shape. When you turn the coin, you will still see the coin as being round. The coin will not appear to change its shape. Nonetheless, there is a sense in which the coin looks different when seen from different angles. We can express this by saying that when you see the coin from a perpendicular angle, it presents a round appearance to you, and when you see it from another angle, it presents an elliptical appearance to you. More generally, things present different shape appearances from different points of view. This phenomenological description is controversial. But, for the purpose of this commentary, I will assume that it is roughly correct.

   The proponents of RPV explain the perspectival character of visual experience by saying that the visual system attributes perspectival properties to the perceived object. Moreover, the proponents of RPV construe these perspectival properties as relational properties. In the case of the coin seen from an angle, for example, the visual system attributes two properties to the coin, namely its intrinsic shape (its roundness) and a relational property. Assaf gives as an example of such a relational property “the property of projecting an elliptical shape on an interposed plane perpendicular to the line of sight” (p. 2). Relational properties of this kind are mind-independent, but viewpoint-relative.

   I would like to make two remarks about the relational properties postulated by the proponents of RPV. My first remark concerns the nature of these properties. I think that the way in which Assaf defines them is somewhat problematic. In ordinary situations of observation, there is no plane between the perceiver’s viewpoint and the object. When you look at the round coin from an angle, the coin does not actually project a shape onto a plane. (As a matter of fact, a plane would hide the coin.) As a consequence, Assaf’s definition characterizes a disposition, namely the coin’s disposition to project an elliptical shape onto a plane perpendicular to the viewer’s line of sight. But this invites a further question, namely how this disposition can account for the perspectival character of the experience.

   Perhaps, it is possible to answer this question. But I think we can also avoid it if we construe perspectival properties in a different way. One option here is to define perspectival properties in terms of outline shapes, as suggested by Robert Hopkins. Hopkins defines an “outline shape at a point as the solid visual angle it subtends at that point” (Hopkins 1998, 55). A solid visual angle is a three-dimensional geometric object that is relative to a point of view. Suppose again that you look at the coin from an angle. The solid visual angle consists of all the straight lines that lead from your point of view to all the points on the edge of the coin. Such an angle is a relational property. It is determined by both the object’s shape and the point of view from which the lines are drawn. Moreover, a visual solid angle does not require a projective plane and is thus not a dispositional property.

   My second remark concerns the way in which the visual system represents these relational properties. We can distinguish between two different options. The visual system could represent these relational properties as relational properties. In this case, the visual system would represent the viewer’s point of view explicitly. Very roughly, one could characterize such a property as looking elliptical from here, or, if we define it in terms of outline shapes, as instantiating visual solid angle α from here. Alternatively, the visual system could represent the relational properties as monadic properties (see, for example, Schellenberg 2008). In this case, the visual system would not represent the viewer’s point of view. Again, very roughly, one could characterize such a property as looking elliptical, or as instantiating visual solid angle α.

    

   I believe that the first option is more plausible. Viewers are usually aware of the point of view from which they look at objects in their visual field. But, more importantly, it seems to me that the second option would not allow us to provide evidence in favor of RPV. Let me explain. According to Assaf, the alternative to RPV is RET. Proponents of RET (e.g. the vision scientist Irvin Rock) account for the perspectival character of perceptual experience in terms of representations of shape properties of the retinal image. Since these properties are intrinsic properties of the retinal image, the visual system would represent them as monadic properties. But how could we then distinguish evidence for or against RET from evidence for or against RPV? First, it seems impossible to distinguish between these two kinds of evidence from a first-person point of view. From this point of view, representations of shape properties of the retinal image would seem exactly like representations of the respective relational properties. This would affect Assaf’s proposed experiments because they are based on first-person responses. Second, it also seems impossible to distinguish between these two types of evidence from a third-person point of view, say, through an investigation of the visual system. As Assaf has argued, a detector, say a neuron, for these properties would respond in the very same way in both cases. We should therefore define RPV as the claim that the visual system represents the relevant relational properties as relational properties, rather than as monadic properties.

   I would like to conclude these preliminaries with a terminological remark. Assaf sometimes states that RET is a version of the sense-datum theory. I believe that many sense-datum theorists would resist this way of speaking. As we have seen, RET states that the visual system constructs representations of certain properties of the retinal image. The sense-datum theory, in contrast, states that sense-data are direct objects of visual experience. The relation between experience and sense-data is usually construed as direct acquaintance, rather than as representation.

    

   2. Can we provide empirical evidence for or against RPV?

   Assaf proposes experiments that would provide evidence for or against RVP based on the object files paradigm introduced by Kahneman, Treisman, and Gibbs. KTG introduce the object files paradigm as follows:

   We assume that the end product of perceptual processing of a stationary scene is a set of object files, each containing information about a particular object in the scene. Each object file is addressed by its location at a particular time, not by any feature or identifying label. It collects the sensory information that has so far been received about that object at that location. This information can be matched to stored descriptions to identify or classify the object, but it need not be (Kahneman, Treisman, Gibbs 1992, 178).

   In other words, the authors suggest that the visual system opens a file, an object file, for each visual object and then enters further information about various sensory features of these objects into their files as it becomes available. KTG hypothesized that the existence of object files would lead to certain advantages in object recognition. Under certain circumstances, features stored in object files should be recognized faster than other features. KTG then designed various experiments that confirmed robust object benefit effects of the following kind: features that were perceived as belonging to numerically identical objects could be recognized faster than features that were perceived as belonging to numerically different objects. Assaf suggests that similar experiments could be used to investigate whether the visual system also enters the perspectival properties postulated by RPV into object files. In the following I would like to address two of Assaf’s proposed experiments.

    

   Assaf’s first experiment is designed to show that the perspectival properties postulated by RPV enter into object files. Assaf describes this experiment as follows. Subjects are first presented with a preview display containing different three-dimensional objects seen from an angle (Figure 1). The subjects are then shown either identical displays (same object condition) or different displays (different object condition) and asked to name the perspectival shape of an object in a certain position. If subjects answer the question statistically faster in the same object condition than in the different object condition, we have evidence for the fact that the visual system enters the relational properties stipulated by RPV into object files. I would like to formulate two questions about this experiment.

    

   
   Figure 1: Assaf’s display for his first experiment

   My first question concerns the role of the relevant perspectival properties. According to KTG’s paradigm, object files are addressed by location. As I understand this, the visual system uses location information to track objects and, hence, to identify object files. As long as an object is in view, information about its location is constantly updated while other information is added to the object file. When the object is temporarily out of sight, say, for example, when it disappears behind a screen and then reappears, the visual system has to integrate information about the location at which the object disappeared with information about the location at which it reappears. Only if the visual system is able to do this, can it maintain one single object file for this object. It follows that information about the location of visual objects plays a different role than information about other sensory features of these objects. Information about location properties allows the visual system to identify an object and maintain a corresponding object file. But object files do not represent information about location properties.¹

   If what I have said in the previous paragraph is true, it follows that we cannot use the KTG paradigm to test whether location properties enter into object files. We can use location information in order to address different object files. It might even be true that the visual system stores location information in object files. But we cannot use the object benefit effect to show that the visual system enters location information into object files. Now, since RPV defines perspectival properties in relation to the viewer’s point of view, they are partly determined by location information. My first question is the following: What reasons do we have for thinking that the visual system treats information about these relational properties in the way in which it treats information about other sensory features, rather than in the way in which it treats information about location properties? Can we design experiments that would allow us to decide between these two options?

   One might be tempted to respond that this question will be answered if experiments designed according to KTG’s paradigm show an object benefit effect with regard to the relational properties postulated by RPV. But this does not seem right. The reason for this is that the interpretation of a possible object benefit effect depends on how we answer this question. Suppose that Assaf’s experiments show an object benefit effect and suppose also that the visual system uses information about relational properties in order to identify visual objects. We could then explain an object benefit effect as a consequence of visual system’s use of that information. But it would not follow that the visual system represents these relational properties as relational properties and stores them in object files.

   My second question is related to my previous point and concerns possible dissimilarities between KTG’s original static display and Assaf’s experimental setup. In their original experiments, KTG begin with a static display. As Assaf explains, in this experiment, subjects first view a preview display. This display contains boxes, some of which contain letters. The subjects then view two different types of displays.² Displays in the same object condition contain the same letter in the target location. Displays in the different object condition contain different letters in the target location. See figure 2. In the static display, neither letters nor positions are perceived to move. This experimental setup allows us to identify object files by the locations of the boxes: identical locations, that is, identical boxes, are associated with identical object files. In other words, the letters are treated as features of numerically identical or numerically different locations.

    

   
   Figure 2: Partial reproduction of figure 1 in (Kahneman, Treisman, and Gibbs 1992)

   Assaf’s display in figure 1 is also a static display. But, in contrast to KTG’s display, it does not contain location markers, such as boxes, and thus presupposes that the relevant object files are individuated by unmarked locations. This has the consequence that we can explain putative object benefit effects in a number of different ways, and, in my view, this renders the experiments inconclusive. I would like to illustrate this with two examples.

   Let us assume that Assaf’s experiment shows a robust difference between the time it takes to name the perspectival shape in the same object condition and the time it takes to do this in the different object condition. One way to explain this difference is to say that the subjects quickly recognize the overall scene in the same object condition as identical with the overall scene in the preview display. We could then plausibly say that subjects can recognize quickly that both scenes instantiate identical perspectival properties because they recognize the two scenes as identical. We do not need to assume that perspectival properties are also represented in object files.

   A second way to explain the difference is based on then assumption that the visual system stores perspectival properties in object files. First, if the visual system stores the relevant relational properties as relational properties in object files, these representations are plausibly updated when the object moves relative to the subject. Second, in Assaf’s display, it is plausible to assume that the subjects perceive the yellow disc in the different object condition as having moved from its original position in the preview display to the position of the square. We could then say that it takes more time to name the perspectival property in the different object condition than in the same object condition because the visual system needs time to update the location information in the object file. As far as I can tell, this is a plausible explanation of the data. But it presupposes, and can thus not show, that the visual system stores perspectival properties in object files.

   Can we modify the experiments in order to get around these difficulties? An obvious modification would be to simply change the objects in both the same object condition and the different object condition in appropriate ways. For example, one could simply exchange the yellow disc on the bottom right with an object that is not part of the preview display. This would render both of these alternative explanations less plausible. Yet, even this modification would not exclude other possible explanations. Suppose the experiment thus modified still shows a robust difference between the time it takes to name the perspectival property in the same object condition and the time it takes to do so in the different object condition. This effect could piggyback on the object benefit effect associated with the object’s objective shape. If a subject can tell the objective shape more quickly, she might also be able to recognize its perspectival shape more quickly, even if this shape is not represented in the object file. I tried to think of alternative experimental setups that could get around these problems. But this seems to be difficult. My second question therefore is this: Is it possible to modify Assaf’s design in a way that ensures proper identification of object files?

   Let me now turn to Assaf’ second experiment. This experiment is designed to show that the perspectival-objective object files hypothesis is true. This is the hypothesis “that object files with information about objective properties contain information about perspectival properties as well. If perspectival properties enter into object files and if the perspectival-objective object files hypothesis is true, it follows that the visual system attributes the respective perspectival properties to external objects, rather than to the retina. Assaf bases his experiment on similar experiments conducted by Behrmann, Zemel, and Mozer (1998). These experiments show an object benefit effect of a different type than the one discussed above: “Comparisons of visual features are more rapid and accurate within a perceptual object than between distinct objects” (Feldman 2007, 818). This allows us to test whether two given features belong to one perceptual object. Assaf therefore suggests corresponding experiments (see figure 3) in which subjects compare intrinsic angles of objects (e.g., angle β in figure 3) with (a) perspectival angles of the same object (e.g., the perspectival angle γ in figure 3) and (b) perspectival angles of different objects (e.g. the perspectival angle α in figure 3). If these experimental setups show an object benefit effect, it is plausible to conclude that the same object files store both information about objective features and perspectival features, and this would provide evidence for the fact that the visual system attributes perspectival properties to external objects.

   
   Figure 3: Assaf’s display for his second experiment

    

   I find it very difficult to make this kind of comparison in an actual three-dimensional scene. It is a bit easier to make the comparison when looking at a picture. But, as Assaf made clear, this is problematic. The fact that judging perspectival angles is such a difficult task suggests that this involves some imaginary activity on the part of the subject. Ne might argue that the subject estimates the perspectival angle by imagining what the intrinsic angle, which is seen in a certain orientation in relation to the subject, would look like if projected onto a plane perpendicular to her line of sight. Such an estimate depends on both an estimate of the intrinsic angle and its position relative to the subject. It would then also follow that the speed with which the subject could perform the imaginary task would depend on the speed with which she could assess the intrinsic angle. And, according to the object files thesis, the latter depends on whether the angle is perceived a belonging to the same object or to a different one. This scenario would thus yield an alternative explanation of any possible object benefit effect. My question then is the following: How can we exclude such an alternative explanation in terms of the subject’s imaginary activity?

    

   References:

   Gordan, R.D. and Irwin, D.E. (1996). What’s in an object file: Evidence from priming studies. Perception and Psychophysics, 58(8), 1260-1277.

   Hopkins, R. (1998). Picture, Image, and Experience. Cambridge: Cambridge University Press.

   Kahneman, D., Treisman, A., and Gibbs, B.J. (1992), The reviewing of object-files: Object-specific integration of information. Cognitive Psychology 24, 175-219.

   Schellenberg. S. (2008). The situation-dependency of perception. Journal of Philosophy 105 (2): 55-84.

    

   ---

   Notes

   1. Gordon and Irvin write for example: “The file does not contain location information; rather, spatial location is used to address the file” (Godan and Irwin 1996, 1261).

   2. Note that this is a simplification.

   Loading...

2. Does a tilted coin look elliptical? According to most philosophers, one only needs to introspect one’s phenomenal experience to give a positive answer to this question. This answer, however, gives rise to an intuitive contradiction, since the coin also is (and looks) round. ‘Being elliptical’, as opposed to ‘being round’, is a perspectival property. But what are perspectival properties (such as looking elliptical) properties of? Prima facie they do not seem to be properties of external objects, since they do not reflect these objects’ objective properties. One possible answer is to say that they are not properties of objects but of sense data (Broad, 1925). Another possible answer is to say both perspectival and objective properties are perceptually experienced as properties of objects, but the latter refer to “how things are”, while the former refer to “how things are from here” (Noë, 2008).

   The route taken in this paper tries to avoid either of these options by suggesting that perspectival properties are truly attributed by the visual system to external objects. Weksler defends the view that perspectival properties are relational properties of external objects (RPV) by appealing to testable predictions in the object files framework.

   First of all, a small worry about the transparency of experience claim (pp. 2-3). Hill’s quote can be decomposed in two aspects: i) when we introspect, the only available objects of our perceptual experience are external objects; ii) since there are no other objects, then appearances (including perspectival properties) have to be attributed to external objects. The second aspect suggests the following reading: when we introspect, we only see appearances as properties of external objects. However, this might not be an accurate account of phenomenology. On introspection, sometimes we take a property to be both a property of external objects and of contextual and/or internal conditions. One example is lightening that seems to depend both on the color of the object and on contextual information such as the object being in the shade. Or to take a non visual example, the experienced sweetness of a cake might depend on whether we have tasted something acid before. Here the experience could be attributed both to the object and to the internal conditions of the taster. My worry is that a restricted interpretation of the transparency claim might make us loose the relational aspect of perspectival properties. This threatens to be the case in the second part of the paper, where perspectival properties are attributed only to external objects.

   In the rest of my comment I focus on the central argument of the paper. I have four main questions about it, and some suggestions for the sketched experiments.

   OSPB and the content of object files

   Weksler appeals to the ‘object reviewing paradigm’ from Kahneman, Treisman and Gibbs (1992) to test whether perspectival properties are stored into object files. In the original experiment, subjects view two objects with different letters inside them. The letters disappear and the objects change location. A letter reappears either in the same object in which it was first viewed (congruent trials), or in the other object (incongruent trials). The task is to name the letter aloud. Subjects are faster to name the letter in congruent trials than in incongruent trials. This effect is called “the object-specific preview benefit” (OSPB). Objects and properties need not be “numerically the same” for the benefit to occur, it is enough that they contain the same property type.

   Does the presence of an OSPB imply that the object file stores perceptual information about the object? It is important to distinguish between the file itself, which is perceptual, and the content of the file that could, but need not, be perceptual. Object files can, for example, store higher-level abstract conceptual information (Gordon & Irwin, 2000). This does not amount to the refuted claim in footnote 6 that only abstract properties can enter into object files. It amounts to the claim that the contents of a file might not be perceptual, but retrieved from long-term memory (but see Mitroff, Scholl & Noles, 2007 for a different opinion). Therefore, even showing the presence of OSPB involving a certain property does not tell us that the property is perceived.

   A second issue is that OSPB might not even perfectly reliably tell us what enters into the contents of a visual object file. This means that OSPBs may in some cases occur for representations that are not in the content of the file. Take for instance the multi-modal OSPB inspired paradigm by Richardson and Kirkham (2004): 6-month-old infants see a box containing a duck and a box containing a bell. The boxes move, and infants hear a quack: they look at the box that previously contained the duck. Does this show that the sound is stored in the visual object file?

   The mere fact that OSPB occurs – a priming-like effect — with respect to some property does not tell us that the property is part of the content of the file, rather than merely associated with the file: it is not obvious that the only possible source of the priming is presence of the property inside the file. For instance, Gao and Scholl (2010) distinguish between ‘true OSPB’ (due to objects) and ‘illusory OSPB’ (due to other factors): we “[…] need to distinguish true OSPBs that arise from actual mid-level visual processing from similar effects ‘illusory OSPBs’ that may mimic this same pattern due only to strategic scanning and rehearsal” (p. 106).

   Two points thus need more clarification here: first, what tells us that the properties that give rise to OSPB are genuinely visually perceived, rather than merely entered into an object-file from some other source?; second, how do we know that the property is stored into an object file, instead of being merely associated with it and causing a priming-like effect in some other way?

   Object-files and consciousness

   Do object files “mediate the conscious experience of persistence of objects” (p. 13)? Some empirical evidence suggests that object files and conscious perception sometimes come apart. In Mitroff et al. (2005), subjects see an ambiguous display, in which objects can be perceived either as streaming through or bouncing off each other. In a test, subjects consciously saw streaming, while OSPBs were observed in the opposite bouncing direction.

   I think we should not hastily conclude that object files are (always) a good guide to conscious experience. A property might be stored in a file without being consciously perceived; or a property might be consciously perceived without being in the content of a file (when it is associated with the file). In the first case, showing that a property can be stored in the content of a file might not solve the question of what a subject consciously experiences. In the second case, the conscious experience of a property might not be enough to show that the property is perceived as a property of an external object, because it is not in the content of the object file, but it is associated with it.

   “Basing on” claim

   Which properties are more basic? The perspectival or the objective ones? Weksler puts forward the hypothesis that “objective segmentation is based on perspectival segmentation (henceforth the “basing claim”)” (p. 16). The reason adduced is that a) the early visual system uses “perspectival information” in order to segment the visual scene into objects, and b) once this segmentation is achieved the scene is segmented into objects having perspectival properties.

   I suspect that b) does not follow from a): the visual system might use apparently perspectival properties to segment the scene into objects without thereby representing objects as instantiating perspectival properties.

   There are several reasons for this. First, the cues used to detect objects and compute their shapes are not obviously perspectival properties in the sense that is used in the philosophical debate at hand. They are rather representations of certain kinds of connected features (T-junctions, L-junctions, arrow-junctions, etc.). Perspectival properties are projections of an “[elliptical] shape on an interposed plane perpendicular to the line of sight” (p. 2), but these cues are not such projections. They are detected by the visual system to produce representations of overlapping surfaces, and not (yet) of three dimensional objects.

   A connected reason is that the properties used by early vision to segment the scene into objects are unconscious and unaccessible to the subject, while the perspectival properties we are interested in are consciously accessible to the subject. These properties could be represented, but only at a low, subpersonal level.

   Third, the argument is based on a constructivist approach to vision (such as Marr’s). However, there is an alternative picture according to which early vision is neither inferential nor representational (e.g. Orlandi, 2014). To achieve object representations, vision might exploit statistical regularities in the environment and it might turn out that it is less costly and statistically more plausible that vision detects objective properties first and (the conscious) perspectival properties are represented starting from the objective properties. In this case properties (including apparently perspectival ones) used by early vision to segment the scene into objects function as brute non-represented causal triggers (and thus function as Pylyshyn’s locations).

   Can this proposal solve the phenomenal contradiction?

   One of the philosophical puzzles raised by the phenomenal experience of perspectival properties is that the viewer experiences the same object both as elliptical and as round, and that this seems somehow contradictory. Does the fact that object-files represent both the perspectival and the objective shape solve the puzzle? Instead, it merely seems to push the phenomenal contradiction into object files, since the same object file would represent the same external object as being both elliptical and round. This suggests that the philosophical problem is not entirely solved by the proposed solution.

   Constructive suggestions on the sketched experiments

   There are some issues with the experimental display page 11. First, the experiment resembles a change detection paradigm that has been proposed by Luck & Vogel (1997) rather than the object-reviewing paradigm for OSPB. We would need a task more closely modeled on the object-reviewing paradigm to test for precisely the OSPB effect. Second, in the current design objective and perspectival properties are confounded, since the same/different objects have the same perspectival and objective properties. So the preview (or change detection) benefit could be due to the objective property, and not to the perspectival property. We need an experimental design that can distinguish between the two kinds of properties (one option could be to use objects with the same objective shape but viewed from different perspectives).

   Also, it might be costly for visual working memory (VWM) to switch from 2D to 3D displays. Luck and Vogel (1997) show that VWM can store up to 16 features bound to 4 objects. The tested properties were: color, shape, orientation, the presence or absence of a ‘gap’. What happens in 3D displays? Either the perspectival shape is counted by the visual system as one feature or as several features of the object:

    

   

    

   Depending on how the object file “counts” the number of features the concrete implementation of the suggested experiment might or might not be successful merely due to VWM limitations.

   In Figure 3 (p. 17) a possible interference might come from the fact that in the given object the distance between α and β is bigger than the distance between β and γ. One would need to compensate for this difference by presenting objects were the distances for angles within an object are the same as the distances for angles between objects.

    

   Let me summarize my main comments and questions. The first comment concerns the claim that OSPB shows which properties are attributed to objects: OSPB effects might be triggered for reasons that do not depend on property attribution to external objects. The second concerns the relation between object files and consciousness: are object files such a reliable guide for conscious experience? The third comment concerns the hypothesis that the properties used for segmentation in early vision are represented as perspectival properties in object files: this might well be false.

   Finally, I expressed the more general worry that the appeal to object files alone might not be enough to solve the ‘phenomenal contradiction’ which has occupied philosophers, since the contradiction would reappear within the content of object-files.

    

   References

   Broad, C. D. (1925). The Mind and its Place in Nature. Routledge and Kegan Paul.

   Feldman, J., & Tremoulet, P. D. (2006). Individuation of visual objects over time. Cognition, 99(2), 131-165.

   Gao, T., & Scholl, B. J. (2010). Are objects required for object-files? Roles of segmentation and spatiotemporal continuity in computing object persistence. Visual Cognition, 18(1), 82-109.

   Gordon, R. D., & Irwin, D. E. (2000). The role of physical and conceptual properties in preserving object continuity. Journal of Experimental Psychology: Learning, Memory, and Cognition, 26(1), 136.

   Kahneman, D., Treisman, A., & Gibbs, B. J. (1992). The reviewing of object files: Object-specific integration of information. Cognitive psychology, 24(2), 175-219.

   Luck, S. J., & Vogel, E. K. (1997). The capacity of visual working memory for features and conjunctions. Nature, 390(6657), 279-281.

   Mitroff, S. R., Scholl, B. J., & Noles, N. S. (2007). Object files can be purely episodic. Perception.

   Mitroff, S. R., Scholl, B. J., & Wynn, K. (2005). The relationship between object files and conscious perception. Cognition, 96(1), 67-92.

   Noë, A. (2008). Précis of action in perception: Philosophy and phenomenological research. Philosophy and Phenomenological Research 76 (3):660–665.

   Orlandi, N. (2014). The Innocent Eye: Why Vision is Not a Cognitive Process. OUP.

   Richardson, D. C., & Kirkham, N. Z. (2004). Multimodal events and moving locations: eye movements of adults and 6-month-olds reveal dynamic spatial indexing. Journal of Experimental Psychology: General, 133(1), 46.

   Loading...

3. Reply to Smortchkova and Jagnow’s commentaries

   I would like to thank the organizers of the conference: John Schwenkler, Nick Byrd and Cameron Buckner. Special thanks go to Jorge Morales for organizing the present session and for locating commentators. I am thankful to the commentators, Joulia Smortchkova and René Jagnow, for raising many interesting challenges as well as constructive suggestions for the project in general and for the suggested experiments in particular.

   I respond here to what I take to be the most significant issues raised by Smortchkova and Jagnow.

   1. The original object files paradigm (Smortchkova and Jagnow)

   Both commentators rightly point out that the experimental setup I suggest in my first experiment is different from the paradigm from KTG, which involves moving boxes containing letters. In order to apply the original paradigm to the perspectival properties case, we need to use displays with moving boxes containing tilted objects. This creates a complication since in this case the perspectival properties are not properties of the object (the box), but properties of something inside the object. I tried to avoid this complication in the paper by avoiding the boxes altogether, but I agree with Smortchkova and Jagnow that I probably should stick to the original KTG setup, in order to avoid certain problems. Doing so amounts to something like the following setup (see figure 1):

   
   Figure 1: Applying the moving boxes paradigm to perspectival properties

   The subject sees a display containing two boxes with tilted objects in them. The tilted objects disappear and the boxes move. Then one of the tilted objects reappears in one of the boxes. In the “same object” condition the titled object matches the object previously appearing in the same box. The “different object” condition the titled object matches the object previously appearing in a different box. An object specific preview benefit (OSPB) with respect to perspectival properties means that recognizing the perspectival shape is faster in the same object condition in comparison with the different object condition. The existence of this effect implies that perspectival properties enter object files (PIF).

   2. Experiencing properties as relational or as monadic and clarification of the structure of the argument (Jagnow)

   Jagnow writes that my first experiment “is designed to show that the perspectival properties postulated by RPV enter into object files. (p. 5, my emphasis)”

   This is a bit inaccurate. The experiment is designed to show that perspectival properties, no matter their nature (that is, no matter whether relational or not), enter into object files. The aim of the experiment is only to support PIF, and PIF in itself is not enough to support RPV. In order to get to RPV we need the independently supported premise EXT. Remember that the argument is structured as follows:

   1. PIF: perspectival properties (be their nature as it may) enter object files. (evidence: first experiment)
   2. EXT: the object files in question are of external objects. (independent evidence)
   3. Thus, the visual system attributes perspectival properties to external objects, which is inconsistent with RET and consistent with RPV.

   This issue is related to the following worry Jagnow voices. Jagnow asks whether I think of RPV as holding that experiences represent perspectival properties as perspectival properties or as monadic properties. He says that the latter possibility “would not allow us to provide evidence in favor of RPV. (p. 3)”. Why? His reason is that if our experience represents perspectival relational properties as monadic then “representations of shape properties of the retinal image would seem exactly like representations of the respective relational properties”, and this, Jagnow claims, “would affect Assaf’s proposed experiments because they are based on first-person responses.” (p. 4)

   Response: In the first experiment (and mutatis mutandis for the second) the subject needs to recognize the perspectival shape of an object. She need not comment on whether it appears relational or monadic. The point of the experiment is to bypass phenomenological (introspective) disputes about whether or not perspectival properties look relational. The subject need only say what the perspectival shape is (say “elliptical”), and if we find an OSPB then PIF is correct.

   3. Object files and high level abstract information (Smortchkova)

   Smortchkova notes that object files can store higher-level abstract conceptual information (Gordon & Irwin 2000), which (she suggests) may not be perceived. And hence “even showing the presence of OSPB involving a certain property does not tell us that the property is perceived. (p. 2)”.

   Response: I grant that the presence of OSPB involving perspectival properties does not show that they are seen. This means that an important issue is left open by my suggested experiments, namely whether perspectival properties are experienced at all. However, if we assume that they are seen (which is what I implicitly do in the paper), the experiments can be used to test RPV and RET. The first experiment tests whether PIF is true. If PIF is true, EXT is true, and perspectival properties are seen, then RET is false, and RPV true.

   4. Are perspectival properties similar to locations, which don’t enter object files? (Jagnow)

   Jagnow notes that according to Irwin and Gordon (1996), locations don’t enter object files, yet they help determine the continuity of object files. Now since the property <having an elliptical projection relative to here> also involves a location (the location of the perceiver), it might be the case that these properties too don’t enter object files, but only help maintaining them (p. 7).

   First response: as far as I understand, the existence of an OSPB with respect to a given property is evidence that this property enters an object file, unless an alternative interpretation of the data can be provided, e.g., in terms of illusory OSPB or derivative OSPB (see below). How have researchers determined that location information does not enter object files? I am not sure, as Gordon and Irwin do not explain this, and the issue is not mentioned in the original KTG paper, but I would imagine that the reason is that they have not find an OSPB with respect to location. Thus, if there is an OSPB with respect to perspectival properties, it follows that they enter object files (assuming that we’ve ruled out explanation in terms of illusory or derivative OSPB).

   Second response: while perspectival properties involve a location, namely the location of the point of view of the observer, this location, from the subject’s point of view, is always the same (in vector terms, it is [0, 0, 0]), hence it does not appear to serve a role in maintaining object continuity (in contrast to locations of seen objects).

   5. Illusory OSPBs (Smortchkova)

   Smortchkova appeals to the findings of Gao and Scholl (2010), who claim that we “[…] need to distinguish true OSPBs that arise from actual mid-level visual processing from similar effects ‘illusory OSPBs’ that may mimic this same pattern due only to strategic scanning and rehearsal” (p. 106). Smortchkova concludes that “the mere fact that OSPB occurs – a priming-like effect – with respect to some property does not tell us that the property is part of the content of the file” (p. 2).

   Response: Gao and Scholl (2010) explicitly suggest a paradigm to test whether an OSPB is true or illusory (pp. 94-97, 106), and it seems straightforwardly applicable to the case of perspectival properties (to save space, I won’t describe the paradigm). So I accept the criticism. What it shows is that the first experiment should be improved in a way that explicitly rules out illusory OSPBs, using Gao and Scholl’s paradigm.

   6. The basing claim and derivative OSPB (Smortchkova and Jagnow)

   In the paper, I discuss favorably what I call “the basing claim”, according to which, roughly, representations of objective properties are computed on the basis of (consciously experienced) perspectival properties (in the paper the focus is on objective segmentation vs. perspectival segmentation, but I will ignore this complication). Smortchkova challenges the basing claim, and I find most of what she says against it plausible, so I agree that the basing claim is (at best) controversial. But what are the consequences of this?

   This issue appears relevant to criticisms both Smortchkova and Jagnow make about the details of my experiment sketches.

   With regard to on the OSPB with respect to perspectival properties in the first experiment, Jagnow writes:

   “This effect could piggyback on the object benefit effect associated with the object’s objective shape. If a subject can tell the objective shape more quickly, she might also be able to recognize its perspectival shape more quickly, even if this shape is not represented in the object file.” (p. 9)

   Smortchkova makes a similar point in p. 4. We can say that the criticism is that the OSPB involving perspectival properties is derived from an OSPB involving objective properties. Hence the former is not a real, or “original” OSPB, but a “derivative” OSPB, hence it does not show that perspectival properties enter object files.

   The criticism assumes that calculation of perspectival properties is based on representations of objective properties. One way to block the objection is therefore to accept the basing claim, namely accept that the dependence is in the other direction: representations of objective properties are based on representations of perspectival ones. But I’ve granted above that the basing claim is controversial. Another way to block the objection is to hold that neither representation is based on the other. Instead, one might propose that both kinds of representations are independently based on early vision representations (retinal images, Marr’s primal sketch). More on this below.

   But even if experienced perspectival properties are based on objective ones, there is a way out, which Smortchkova herself suggests. She suggests improving the experiment setup by using “objects with the same objective shape but viewed from different perspectives” (p. 4).

   Here is how I understand this promising suggestion (see figure 2): In the preview display we put two objects with the same objective shape (rectangle) but different perspectival shapes. Consequently, in the same object and different object conditions we will get an OSPB with respect to objective shape (rectangle), hence there will be no advantage to the same object condition in comparison to the different object condition, with respect to objective properties. Thus, if we find an OSPB with respect to perspectival properties, we can be sure that this is not a derivative OSPB, i.e., it does not piggyback on an OSPB with respect to objective properties.

   
   Figure 2: Objective properties in the preview and target displays are the same. The difference is only in perspectival properties.

   7. Recognizing perspectival properties via imagination (Jagnow) 

   In the second experiment I suggest in the paper, the task is to compare objective and perspectival angles, (a) of the same object and (b) of different objects. I rely on Feldman’s (2007) claim that “[c]omparisons of visual features are more rapid and accurate within a perceptual object than between distinct objects” (p. 818)”. Thus, if the comparison is faster in case (a) then the visual system treats perspectival and objective properties as properties of the same object, which supports EXT. Jagnow presents an objection, suggesting that

   “the subject estimates the perspectival angle by imagining what the intrinsic angle, which is seen in a certain orientation in relation to the subject, would look like if projected onto a plane perpendicular to her line of sight. Such an estimate depends on both an estimate of the intrinsic angle and its position relative to the subject. It would then also follow that the speed with which the subject could perform the imaginary task would depend on the speed with which she could assess the intrinsic angle. And, according to the object files thesis, the latter depends on whether the angle is perceived a belonging to the same object or to a different one. This scenario would thus yield an alternative explanation of any possible object benefit effect.” (p. 11)

   Here is how I understand Jagnow’s alternative explanation. He appears to rely on the idea that recognizing two features is faster when the two features are of a single object. This object benefit effect is different from those I have discussed in the paper, but it apparently exists (see Duncan 1984). If this effect is indeed real, then recognizing two objective angles of a single object will be faster than recognizing two objective angles, when each belongs to a different object. Now if calculating perspectival angles is based on representations of objective angles, then comparing perspectival and objective angles within a single object will be faster than between two objects, even if the visual system does not attribute perspectival and objective angles to the same object.

   I think this is a good criticism. It is not clear to me whether it is possible to find a “trick” to get around it in a way analogous to Smortchkova’s suggestion regarding the first experiment (any ideas?). So, at present, I think the best bet for me is to deny that representations of perspectival properties are calculated on the basis of objective properties. One way to do this is to accept the aforementioned basing claim (which is problematic in ways Smortchkova describes). A different way is to hold that representations of perspectival properties are directly based on 2D representations in early vision (the primal sketch, the retinal image), and are not based on representations of objective properties (I call this option “DP” in the paper, and I argue against it there). I will try to explain why I currently find this idea appealing by criticizing Jagnow’s positive suggestion about the use of imagination in calculating perspectival angles.

   Jagnow suggests that in trying to assess a perspectival angle we imagine “what the intrinsic angle, which is seen in a certain orientation in relation to the subject, would look like if projected onto a plane perpendicular to her line of sight.” (p. 11). This suggestion feels too “third personal” to me. On this suggestion, as I understand it, when we try to recognize a perspectival angle of a tilted rectangle, such as the acute angle β in figure 3, we first recognize the corresponding objective angle, which is a right angle. Next, we mentally tilt it relative to an imagined plane, draw imaginary lines from it to a point behind the plane, and then “see” (in the imagery) that the angle appearing on the plane is acute.

   There is a different way, involving imagination, to recognize that the perspectival angle β is acute. Raise your finger and use it to trace the contours of the angle, as if you were drawing on a glass window, perpendicular to your line of sight and interposed between the figure and yourself, along the red arrows in figure 3. When you do this you realize that you are drawing an acute angle on this imagined window. Hence you learn that the perspectival angle is acute. The important point is that when you use this method, you are not relying on your knowledge of the objective angle. You can get the perspectival angle correctly, using this method, even if you are wrong or agnostic about the objective angle.

   Perhaps when we try to estimate a perspectival angle, we imagine moving our finger along the contours of a shape in this way (a sort of motor imagery). This seems on the right track to me, as it is first personal and it appears to resemble the way painters recognize perspectival properties. If this is right, then recognizing perspectival angles is not based on recognizing objective angles. I of course grant that this suggestion is speculative and underdeveloped. Does it sound right to you?

   
   Figure 3: recognizing a perspectival angle via moving a finger on an imaginary transparent plane, tracing its contours.

   8. Object files and conscious experience of persistence (Smortchkova)

   Smortchkova discusses experiments showing that persistence for object files could be different from consciously experienced persistence (Mitroff et al. 2005). Smortchkova draws from this the conclusion that a “property might be stored in a file without being consciously perceived” (p. 3). I could be missing something here, but my impression is that the conclusion does not (or at least does not straightforwardly) follow from the premise. As I understand it, Mitroff et al.’s experiments show that, when several objects disappear behind an occluder and then reappear, an object can consciously appear to us to move in one direction while for the object files system it moves in a different direction. This means that the way objects move, “from the perspective of” the object files system, is unconscious. In other words, object persistence, as it is tracked by the object files system, is unconscious.  As far as I can tell, this does not show that a feature stored in the file (like color, shape, etc.) is not consciously perceived.

   9. Visual solid angle instead of projections (Jagnow)?

   In the paper, the example of a relational perspectival property I use is that of having an elliptical projection in relation to here. Jagnow suggests replacing it with the property of subtending a certain visual solid angle. Christopher Hill (2009) has a similar view. It seems to me that the suggestion does not work, although I might be missing something, so I welcome corrections. An ellipse viewed straight on subtends the same visual solid angle as a certain circular coin viewed obliquely. Thus, the visual solid angle in question cannot account for the fact that we experience an elliptical perspectival shape, rather than a circular perspectival shape. It seems that we need to add a plane perpendicular to the line of sight in order to get the ellipse. Perhaps it is possible to somehow add the plane to the visual solid angle story without making it equivalent to the projection story, but I’m not sure how.

   Loading...

4. Hi Assaf, thanks for this very interesting paper. As you know I am a thoroughgoing skeptic about perspectival properties, though I think you make a good case that if there are any such things then their nature can be adjudicated in the way you suggest. I wonder, though, whether you think the empirical paradigms you discuss, or any others for that matter, could bear on the question whether we should believe in p-properties in the first place …

   Loading...

5. Hi Assaf,
   thanks for the paper and the discussion.
   I have one doubt that seems me to be related to Joulia’s worry and more precisely to your reply to her (in 3).
   Imagine that we find out that perspectival properties do not enter object files. Does this mean that we do not perceive perspectival properties? I am not sure, because it might be the case that we do perceive those properties but they do not enter object files. Why should we think that all perceived properties enter into the object file? (apologizes if I have missed the justification)
   I would like also to follow up on John’s question and return his question (if I may). If Assaf reasoning is correct, and we make the experiment, and the result suggests that perspectival properties enter into object files, how would a skeptic about perspectival properties react to such result?

   Loading...

   1. Hi Miguel, this is a good question. Two thoughts: (1) It could be that, as Assaf suggests, experiments like these would show only that perspectival elements are represented in a certain way in the visual system (which is okay by my lights), without anything following directly about their being experienced in ordinary perception. (2) It also could be that, as I suggest in replying to Assaf below, the experience of perspectival elements is partly imaginative, and not purely perceptual, which again is okay by my lights.

      That said, I am enough of an empiricist to evaluate the truth of a hypothesis by its experimental tractability, so as I said to Assaf his paper does significantly more to make me sympathetic to perspectivalism than most of the various introspectionist rationales I’ve encountered for it.

      Loading...

6. Hi Assaf,

   great reply! It clarifies a lot. But I am still puzzled by the “basing on” discussion: it seems that there are two senses of “being based on” at stake now. The first sense (the one in the paper) is how the visual system builds representations; the second sense (the one in the reply?) is how subjects recognize a property. Maybe I am missing something, but could you please explain the relation between these two different issues a bit more?

   (I was also thinking that a possible pre-test would be to simply ask subjects to name the shapes of objects seen under different perspectives and check whether the subjects are faster to name the perspectival or the objective property (this maybe after explaining the subjects what an objective and a perspectival property are). This might bear on the recognition issue.).

   I must add that when I started to read the paper I had a skeptical attitude toward perspectival properties as well, and after reading it I am less of a skeptic. I am very interested in reading your reply to John.

   Thanks!

   Loading...

   1. Hi Joulia and René

      This is a response to Joulia’s question, but it revisits René’s commentary.

      Thanks, Joulia, for another challenging comment!

      I think I see what you find puzzling. In my response to Jagnow (section 7), I discuss recognition of perspectival properties via imagination, as he does. My aim there is to offer an alternative (hopefully a better alternative) to his suggestion on which this recognition-by-imagination is based on objective properties. But, as you say, this kind of “basing” is different from the kind of basing that is the focus of the paper, namely a sort of inferential relation within perception itself, between two kinds of purely perceptual representation.

      So how are these two issues (recognition-by-imagination of perspectival properties and purely perceptual representations of perspectival properties) related? This is an important question, and I didn’t have it clearly in mind before you asked it. So thanks for asking it. Let me try to answer.

      Jagnow’s story about recognition-by-imagination is meant to be a criticism of the second experiment. The criticism is that we find an object benefit effect with respect to perspectival properties because we recognize perspectival properties by imagining what the objective properties we see would project on a certain plane (the evidence Jagnow presents for this is that recognize perspectival angles is difficult). This makes the (Feldman style) object benefit effect derivative. On this story (as I think of it now), perspectival properties are not experienced in a purely perceptual way. Instead, the claim is that they are imagined. One way to understand this is as saying that we experience perspectival properties, but the experience is not purely perceptual, as it involves an imaginative component (perhaps this is similar to John’s suggestion). A different way to understand this is to hold that we don’t recognize a perspectival property by simply taking experience (whether purely perceptual or not) at face value. Instead, experience (purely perceptual or otherwise) only presents objective properties, and we need to use imagination in order to go beyond experience and recognize a perspectival property. On such a story we do not experience perspectival properties at all (not even impurely). Given Jagnow’s aim, namely to undermine my second experiment, this difference between the two readings does not matter.

      I have borrowed Jagnow’s formulation in my response to him, so these issues come up again in my recognition-by-imagination example. I must reject both sorts of readings. On my view (more precisely, on the view presupposed by the paper), we have purely perceptual experiences of perspectival properties (let me put aside the question of whether they are based on objective properties or not). If so, why do we need to imagine tracing the contours of the perspectival angle with our finger? Why can’t we simply take the experience of perspectival properties at face value?

      Here is a possible (again speculative) answer. Let’s return to Jagnow’s claim that recognizing a perspectival angle is difficult. Why is it difficult given that we experience perspectival properties (in a purely perceptual way)? Suggestion: it is difficult because the experience of the objective angle interferes. Perhaps this resembles the Stroop effect: when we attend to a perspectival angle, we involuntarily attend also to the overlapping objective angle (this should be expected if the visual system attributes both properties to the same object). So perhaps in the same way it is difficult to tell what the color of a word is, if the word means a different color (the Stroop effect), it is difficult to name a perspectival angle given that the (overlapping) objective angle is different. So perhaps imagining hand movements along the contours of the perspectival angle somehow helps highlight the perspectival angle, thereby diminishing the influence of the objective angle.

      Loading...

      1. Hi Assaf,

         a short follow up. If I understand your exchange with René correctly, a problem for you is the case in which objective shapes are purely perceived, while perspectival properties are not purely perceived but need imagination to be experienced. But one could say that even for the objective 3-D shape we experience it with the help of imagination (to imagine the backside of the object for example). So I am not sure whether presence or absence of imagination could help us drawing a line between the way we perceive objective properties and the way we perceive perspectival properties. What do you think?

         Loading...

7. Thanks, Assaf, for a very interesting paper. I find your experimental proposals very promising. However, I want to register a further doubt concerning the “basing claim.” (Apologies if I am simply reiterating a point that has already been made.)

   You note, following Feldman (2007), that retinal image elements tend often to be perceptually grouped on the basis of certain “special” geometrical relations like collinearity, cotermination, parallelism, etc. You further suggest that we can construe the visual system as relying here on “perspectival” properties of distal objects (e.g., the property of projecting collinear images onto the retina). You then write:

   “A plausible way to interpret this idea, given the object files framework and given the truth of PIF, is by suggesting that there are processes that create objects files containing – and on the basis of – representations of perspectival information, subsequently inserting into these files representations of objective properties, while updating the segmentations (i.e., splitting and merging files) if needed.”

   I don’t know how plausible this is, for the following reason. The perspectival relations that Feldman examines (e.g., projecting collinear, coterminating, or parallel line segments onto the retina) are all highly informative concerning corresponding objective relations in distal 3-D space. Thus, two visible lines that are collinear in 3-D will always (discounting noise) project collinear lines on the retina, and the probability that two non-collinear lines in 3-D will project collinear lines on the retina is very small (e.g., the edges of two surfaces would have to be precisely “lined up” from your perspective). As such, the detection of such properties on the retina is often sufficient for inferring the presence of corresponding properties in 3-D space (Biederman 1987). But, given this, it is unclear why we should think that the visual system encodes these perspectival properties in object files, instead of simply discarding them in favor of the corresponding derived 3-D properties (e.g., collinear in 3-D, parallel in 3-D, etc.). Indeed, storing both kinds of properties may simply be redundant, unless there is some special reason for retaining the perspectival information. Thus, I think more needs to be said in order to make the proposal quoted above plausible.

   Note, moreover, that the position I’m suggesting would not entail commitment to the “two sets of object files” hypothesis that I think you plausibly dismiss. The representation of the perspectival property (or 2-D image property, depending on how we construe the computation) is on this view simply replaced, rather than stored somewhere else.

   Loading...

   1. Hi E.J.,

      Thanks for an interesting challenge!

      You move a bit quickly, so let me try to reconstruct your objection slowly. You focus on the fact that 3D (objective) collinearity, cotermination, etc., are easily recoverable from their perspectival (2D) counterparts, and vice versa. Indeed, as you present this, perspectival colliearity (cotermination, etc.) implies 3D collinearity, and vice versa. It seems that no calculation is required at all. The visual system need only change the label of the representation from 2D (perspectival) to 3D (objective).

      Though you don’t say so explicitly, I assume your point is to contrast this with more familiar cases of inferring objective information from perspectival information. For example, a perspectival ellipse can be interpreted as an objective ellipse or as an objective titled circle (coin), and also, an objective circle can have (given different viewpoints) different perspectival properties (a circle, an ellipse). Thus, recovering a 3D (objective) property from its perspectival counterpart (or vice versa), is difficult, and requires substantive calculations.

      What role exactly does this contrast (between easy and difficult perspectival->objective inferences) play in you objection? It seems to be this: if the visual system can easily move from perspectival information to objective information without performing any calculation (i.e., mere label changing), then it is redundant to retain the perspectival information in memory (hence in object files). Your stressing easy inference (or mere label changing) suggests that your think that if the transition from perspectival cotermination (colinearity, etc.) to 3D (objective) cotermination were not easy, then it would be useful to retain perspectival cotermination in object files (call this the Difficult->Useful conditional).

      Is this reconstruction of you objection correct? Some things you say (specifically, your “unless there is some special reason for retaining the perspectival information”) suggest you don’t accept the Difficult->Useful conditional. But if so, then it is not clear to me why you emphasize the fact that the transition between perspectival and objective cotermination (etc.) is easy. So if my reconstruction is incorrect, could you please explain what role the aforementioned contrast (between easy and difficult perspectival->objective inferences) plays in your objection, if any?

      Now, if you grant the Difficult->Useful conditional, then you grant that perspectival cotermination information is useful, but you suggest there is no need to store it in memory (in object files), because the objective (3D) cotermination information can de facto play the role of perspectival cotermination information in any subsequent calculation, because the two sorts of information perfectly match each other.

      If this is the suggestion, then it appears to imply that the visual system creates an ambiguity: a certain representation, say of cotermination, represents (hence is ambiguous between) two different properties, namely perspectival and objective cotermination. Is it plausible that representations in the visual system are ambiguous in this way? I don’t know. But this suggests a strategy for answering your objection, namely to argue that it is not redundant to store perspectival cotermination information in object files, because doing so avoids the ambiguity in question.

      Does this make sense to you?

      Loading...

      1. Hi Assaf,

         Thanks for your insightful response. I wouldn’t want to endorse the Difficult -> Useful conditional. I’m not sure that the difficulty of deriving objective information from certain perspectival information bears any direct relation to the usefulness of that perspectival information. (There may be such a relation, but if there is, it’s not immediately obvious to me.)

         I’m inclined to think the usefulness of perspectival information should be evaluated on independent grounds (e.g., would this information enable the organism to solve certain important problems, etc.). My idea was that in this case it’s not clear what usefulness the relevant perspectival information would have, while the corresponding objective properties are more obviously ecologically significant.

         The role of the easiness of the derivation was the following: Given that the computation is “easy,” it seems the visual system is in a position to extract much more useful information. In contrast, if the derivation were very difficult — e.g., extracting precise depth values — then it might be reasonable to hold that the visual system stores perspectival information just because it is not in a position to recover the corresponding objective information.

         Is this any clearer?

         Loading...

8. Dear Assaf,

   Thanks so much for your interesting paper and your detailed response to my commentary, including the many clarifications. I have one follow-up question that is related to the modified set-up of the first experiment. One worry about the original set up was that an OSPB involving perspectival properties could derive from an OSPB involving objective properties. You made two modifications to your original experimental set-up: you use a dynamic display, that is, you use boxes that change their position, and you adopt Joulia’s proposal of using objects with the same objective shape but viewed from different perspectives. You respond to the worry that an OSPB involving perspectival properties could derive from an OSPB involving objective properties as follows:

   “In the preview display we put two objects with the same objective shape (rectangle) but different perspectival shapes. Consequently, in the same object and different object conditions we will get an OSPB with respect to objective shape (rectangle), hence there will be no advantage to the same object condition in comparison to the different object condition, with respect to objective properties. Thus, if we find an OSPB with respect to perspectival properties, we can be sure that this is not a derivative OSPB, i.e., it does not piggyback on an OSPB with respect to objective properties.”

   I have a comment and then a question about your modified set-up. Since the boxes move, the objects would actually change their perspectival properties. It seems to me that we can take care of this by making sure that this difference is negligible in comparison to the difference between the perspectival properties instantiated by the two different objects in the preview display. If this sounds right, I have the following question: Could we not explain a possible OSPB as the result of the fact that in the same object condition the visual system represents an object with the same shape in (almost) the same orientation and in the different object condition the visual system represents an object with the same shape in a different orientation? The underlying assumption here would be that the visual system needs more time to compute the different orientation and the subject would thus need more time to name the perspectival shape. We can also give a similar explanation of a possible OSPB that does not appeal to the objects’ orientation relative to the viewer. Since the objects are now in boxes, a possible OSPB may be the result of the fact that in the same object condition the spatial relations between object and box are identical and in the different object condition they are different. It seems to me that this may also have the effect that the subject could name the perspectival shape faster in the same object condition than in the different object condition. I am wondering what you would say about this.

   Loading...

   1. Dear René,

      Thank you taking the time to raise yet another important and illuminating issue.

      Let me try to address the problem you raise vis-à-vis derived OSPB and orientation. I am quite convinced that it successfully shows that the modified experiment I suggest does not yet remove the “derived OSPB” worry. I need to think about it more, but here is an initial response:

      Perhaps we can compare the observed OSPB with respect to the (objective) orientation to the observed OSPB with respect to the perspectival shape (I will clarify my emphasis on “observation” below). Assuming (as your objection suggests) that calculation of perspectival shape is partly based on the representation of orientation, we expect that, if there is a non-derived OSPB with respect to perspectival shapes, then the observed OSPB with respect to them (perspectival shapes) will be bigger than with respect to objective orientation.

      Let me try to clarify this idea by using a toy example. Suppose the time it takes (without OSPB) for the visual system to create (infer) a representation of objective orientation is 500ms. Suppose the time it takes (without OSPB) for this system to create (infer) a representation of perspectival shape on the basis of objective orientation is 20ms. So it’s 520ms for the full (two-part) calculation (without OSPB). If there is a non-derived OSPB with respect to perspectival shapes, then the second calculation will be faster, say 15ms (the preview benefit is 5ms). If there is a non-derived OSPB with respect to objective orientation, then the first calculation will also be faster, say 480 ms (the preview benefit is 20ms). Overall, the full calculation, given the two non-derived OSPBs, is 495ms. The combined preview benefit is 25ms. Thus, when doing the experiments, the observed OSPB for objective orientation will be 20ms (the actual OSPB is also 20ms), whereas the observed OSPB for perspectival shape will be 25ms (the actual OSPB is 5ms). Thus, if in the experiment we find that OSPB with respect to perspectival shape is bigger than the OSPB with respect to objective orientation, this is evidence that there is a non-derived OSPB with respect to perspectival shape.

      What do you think?

      Loading...

      1. Hi Assaf,

         Thanks again for your response! As far as I can tell, this is very plausible. If the two experiments work out in such a way that the observed OSPB with respect to perspectival shapes is bigger than the observed OSPB with respect to objective orientation, it seems plausible to conclude that the former is not derived from the latter. Do you think that we could draw the same conclusion if we would find that the observed OSPB with respect to perspectival shapes is smaller (to a certain minimal degree) than the OSPB with respect to objective orientation? In any case, I am becoming increasingly curious about the possible results of your suggested experiments.

         Loading...

9. Hi Miguel

   Thank you for taking the time to look into my paper and for your comment, which requires that I clarify things some more.

   You ask about the consequences of perspectival properties not entering object files. Does this imply that perspectival properties are not experienced? Well, no, or not so quickly. One possibility is that they are indeed not experienced. Another possibility is that, as you suggest, they are experienced but do not enter object files. In this case we should explain why they don’t enter the files.

   One possible explanation is that they are somehow like locations, which also don’t enter files (Jagnow makes a suggestion about this in his commentary, and I have responded in section 4 of my response to the commentaries). Perhaps something in the spirit of what E.J. Green says in his comment can help here. It might be the case that the object files system uses perspectival properties in order to calculate objective properties (which it stores in files) but then discards the perspectival properties, perhaps because, as Green suggests, in some cases (of collinearity, cotermination, etc.) it is redundant to store both kinds of properties, or perhaps because (this is a very sketchy speculation) the aim of the object files system is to store only objective properties of objects (because, e.g., these are more relevant to action, hence survival, etc.).

   An alternative explanation is that perspectival properties do not enter object files because the visual system attributes them to inner items (e.g., regions in the retina) and not to external objects. In roughly this way we can argue from the premise that perspectival properties don’t enter object files to the truth of RET (and falsity of RPV). This is what I had in mind in the original paper, where I write that the falsity of PIF counts against RPV.

   Why think that features the visual system attributes to inner items don’t enter object files? One reason is that arguably the visual system is not designed to track inner items, but only outer items (see the end of section 3 in my paper).

   I turn to your response to John’s question. Note that John asks why we should believe that perspectival properties exist, whereas I assume that they exist, and focus on the question whether we perceptually experience them. As far as I understand, this is merely a difference in terminology.

   Suppose perspectival properties enter object files. Does this imply that perspectival properties are perceptually experienced? Joulia answers in the negative in her commentary (section “OSPB and the content of object files”), and I agree. As she points out, object files can store conceptual, high-level, abstract information, such as categorical information, information about meanings of seen words, letter type (independent of case), and so on. Arguably, some of this information is not perceptually experienced (or at least, whether we perceptually experience it is controversial). Thus, the claim that a feature enters an object files, in and of itself, does not entail that the feature is perceptually experienced.

   Loading...

10. Hi John (and Miguel and Joulia who expressed interest in John’s question),

    I have a sketch of an argument aiming to show that we experience perspectival properties (or, if we use your terminology, the argument shows that perspectival properties exist. John, I hope I haven’t mixed-up something here). I haven’t thought about it thoroughly, so it probably has many holes. If you can spot them it would be cool. Also, although it involves an empirical finding, in the end (unfortunately) it relies on intuition (or introspection), and so it is probably what you asked for (namely a purely experimental paradigm).

    The argument relies on a certain version of the hallway illusion. The illusion, as it is presented by Palmer (1999, see below figure 7.1.2 from his book together with Palmer’s caption), is an illusion about perspectival properties, although Palmer himself (see the caption) is not clear enough about this (his talk about size is often ambiguous between perspectival and objective size).

    Consider figure 7.1.2a. Palmer says that while the farthest cylinder looks smaller than the closest cylinder, “most people are astonished that the difference is as large as it actually is” (caption to figure 7.1.2). The way I understand this, and experience this, is that the farthest cylinder does not look objectively smaller than the closest cylinder. All cylinders look to have the same objective height. However, the perspectival size (height) of the farthest cylinder looks smaller than that of the closest cylinder. Now, as Palmer says, I am surprised to learn that the perspectival size of the farthest cylinder it is as small as the (perspectival size of the) black cylinder. Thus, the perspectival size of the farthest cylinder looks bigger than it actually is. Thus, the present case is a visual illusion with respect to perspectival properties.

    I now add the premise that a visual illusion with respect to a property implies (or at least suggests?) that the property is visually experienced. This appears to be so with the usual illusions: the Muller-Lyer illusion is about length, implying that length is experienced. We have illusions about depth (3D glasses, the mask illusion), implying that depth is visually experienced. We have visual illusions about motion (the waterfall illusion), implying that motion is visually experienced. Thus we have:

    there are visual illusions with respect to perspectival properties
    if there is a visual illusions with respect to a property P, then we visually experience P
    Therefore, we visually experience perspectival properties

    One might object to 1, saying that the hallway illusion might not be a genuine visual illusion (unlike the other illusions mentioned). Perhaps we do not experience perspectival properties at all, and the illusion is only an issue of judgment, imagination, calculation, or other cognitive activity that goes beyond what is given in experience. For example, perhaps we try to assess what the perspectival size of the farthest cylinder is, by actively calculating what an object with its objective size would project on a plane perpendicular to the line of sight. And we get it wrong, perhaps because we are influenced by the experience of all the cylinders as having equal objective size.

    In response I note that the hallway illusion feels like a genuine visual illusion to me. In support of this we can consider cases that clearly involve active calculation: I try to assess (calculate, imagine) the number of candy bars in a jar on the basis of looking, without counting each one, and I get it wrong. This does not feel at all like a visual illusion. Or consider trying to estimate (calculate, imagine holding it in your hand) the weight of a piece of cheese on the basis of the way it looks, and getting it wrong. Again, it does not feel at all like a visual illusion. Lastly, consider trying to calculate (imagine) what a certain tilted object projects on the ceiling and getting it wrong. This also does not feel like a visual illusion at all.

    What do you think? What am I missing?

    Loading...

    1. Hi Assaf, this is a really interesting case! And I have to think more about it. But right now I suspect the argument could be resisted by claiming that the illusion you describe is not at the level of ordinary perception, but rather that of what Robert Briscoe (2008, 2011, forthcoming) calls “make-perceive”, or the top-down imaginative construction of — in this case — a visual image corresponding to the shape that would be needed perfectly to occlude a given object. I think it’s uncontroversial that we can do this, and that make-perceived shapes, etc. are available to introspection, but my hunch is that when you introspect perspectival shapes and sizes in the paradigm you describe what’s really happening is that you’re first make-perceiving them, then introspecting these products of your imagination. At least, I think it’s possible that you’re doing that, and that findings like this one don’t settle which story is the right one.

       How does this sound to you?

       Loading...

       1. Hi John,

          This requires more thought, but here is an initial (underdeveloped, etc.) line of argument against your suggestion.

          Briscoe’s (2008, 2011) central examples of “make perceive” outside the context of perspectival properties are make-perceiving a constellation in the night’s sky, make-perceiving a sofa located at the corner of a seen empty room (when trying to decide if it will fit), and make-perceiving a seen cat’s occluded tail (a sort of amodal perception). None of these examples appear to involve illusions.

          So now we can compare two accounts of the hallway illusion. On the first, perspectivalist account, this is an ordinary perceptual size illusion, like many others we are familiar with (the Ames room, Muller-Lyer, Ebbinghaus, Ponzo etc.). In contrast, on yours and Briscoe’s “make perceive” account, it is an illusion on the level of make-perceive. Given that we are not familiar with other illusions on the level of make-perceive, the first account looks better. In other words, it is prima facie implausible to hold that the only illusion on the level of make-perceive is the one involving perspectival properties.

          Relatedly (although this is perhaps a somewhat different line of argument), why don’t we experience a “make-perceive” illusion when we imagine what a cylinder would project on the ceiling, or on a sphere?

          This can be thought of as two challenges to yours and Briscoe’s account: (a) locate an example of an illusion at the level of make-perceive, outside the context of perspectival properties; (b) explain why we have illusions with respect to projections on a plane perpendicular to the line of sight, but not with respect to projections on the ceiling or on a sphere, despite the fact that we can easily imagine them.

          Does this make sense?

          Loading...

       2. Hi Assaf, thanks for this reply. I have to think more about this, but it seems plausible that there would be illusory make-perceiving if e.g. I have to determine whether two line segments in the Ponzo illusion are the same length:

          

          Here, I will make-perceive a superimposition of the target line over the others (or, perhaps, make-perceive two lines running perpendicularly from the sides of it), and that make-perception will be illusory.

          Does this fit the description of what you are looking for? I will let you have the last word (for now)!

          Loading...

11. Oops! In the previous comment I tried to embed an image and it didn’t work (figure 7.1.2. from Palmer). So please use this link:
    http://gdurl.com/TUAN

    Loading...

12. Hi John (this is a reply to your Ponzo illusion comment),

    A nice rejoinder!

    I think there are two ways to understand your suggestion. On one interpretation, you suggest that the Ponzo illusion is a “make-perceive” illusion, and hence not a genuine (pure) perceptual illusion. My initial response is to suggest that the Ponzo illusion is too similar to the hallway illusion (a hallway stretching into the horizon vs. rails stretching into the horizon, both leading to a size illusion), and so the Ponzo illusion could be interpreted as an illusion about perspectival lengths arising from depth cues (or as something closely related to that), hence it does not answer the challenge (of finding an illusory make-perceiving of something other than perspectival properties).

    A second way to understand your suggestion is as granting that the Ponzo illusion is a genuine (pure) perceptual illusion, and as holding that this perceptual illusion gives rise to a corresponding make-perceive illusion. If this is the idea, then in order to avoid the preceding problem, you can choose an illusion that is clearly not about perspectival properties, but about objective properties, like, e.g., the Ames room. One can describe a case of looking at the Ames room and imaginatively “moving” one person in 3D, superimposing it on a different person in the other side of the room, in 3D. This make-perceiving is illusory, and is not about perspectival properties. It therefore plausibly answers my challenge.

    However, on this story, the illusory make-perceiving is derivative. It is not the source of the illusion. According to this story, the Ponzo illusion is purely perceptual, and the make-perceiving illusion is a “copy” of the perceptual illusion. In contrast, the hallway illusion, on your account, depends on imagery. The illusion does exist on the purely perceptual level.

    I thus update the challenge: it is to find an illusion on the level of make-perceiving that (a) is not about perspectival properties, and (b) is not derived from (i.e., is not a “copy” of) a purely perceptual illusion.

    You have my permission to have the last word (it is 12:49am here, so for me the session is pretty much over)

    Loading...

13. Hi Joulia (this is a response to your comment about imagination and experiencing the back side of objects),

    Thanks for this!

    In this response I clarify why (and that) I think it is problematic for a believer in perspectival properties to hold that our experience of perspectival properties is based on imagination. It is problematic, I think, because our imagination is too unconstrained. When looking at a tilted square, instead of imagining what it would project on a plane perpendicular to the line of sight, we can imagine the shape it would project on a sphere. But intuitively, the square does not look to have that shape. Put differently, perspectival properties are supposed to be different from properties like “projects such-and-such on a sphere”, in that the former are in some sense really experienced, whereas the latter are not. Adding imagination blurs this difference.

    Perhaps it is possible to say in response that the imagery involved in experiencing perspectival properties has some special feature, which the imagery of “projection on a sphere” lacks. One could then try to use this feature to explain the sense in which perspectival properties are really perceived, while “projecting X on a sphere” is not.

    Loading...

Comments are closed.