Edinburgh Research Explorer Is augmented reality a source of new types of knowledge?

: Some everyday cases of cognition show how computers functioning within Plain Reality give us a new type of knowledge. In contrast, the project of Augmented Reality is epistemologically challenging because it proposes hybrid scenarios which are friendly for cognitive agencies but infuse them with Virtual Reality (VR) overlay that is alienated from reality. Working with the as-sumption that Augmented Reality is ontologically heterogeneous, as it mixes experiences of individual objects with experiences of models, we examine its cognitive usefulness.We argue that insofar as our cognitive contact with Augmented Reality-based simulations may be even better than the celebrated contact with reality, there is room for extending the notion of knowledge.


Introduction
Pieces of knowledge-that mayb ei nterpreted as products of cognitivea gents who, for example, transformtheir veridical experiencesinto true beliefs. Skeptical "Brain in aV at" (BIV) scenario suggests thatabrain totallyi mmersed into Virtual Reality (VR) stands no chance of comingt ok now because its agency and experiences become illusory.Wequestion whether BIV scenario is consistent if it does not explain how the agency of abrain can be reconstructed on the basis of VR experiences.Granting that agency and "contact with reality" are standard conditions for knowledge-that to emerge,w ea sk what is required to create an environment for an ew type of experience which could resulti nan ew type of knowledge.S ucha ne nvironment should not violate cognitive agency and offer anew type of non-deceptive experience. There are everydaycases of cognition which show how computers functioningwithin Plain Reality provide us with information which is anew type of knowledge.These cases are easilyacceptable because the information respects the requirement that we remain in touchwith reality.I nc ontrast, the project of Augmented Reality is epistemologicallyc hallengingb ecause it proposes hybrid scenarios which are friendlyf or cognitive agencies but infuse them with VR overlaythatisalienated from reality.Augmented Reality scenarios do not tamper with our senses and essentiallyextend of our experiences. They provoke, however,t he question in what sense VR overlay could be veridical, as it somehowm ustb ei no rder to support instances of new types of knowledge.Wepropose thatitishelpful to assume that Augmented Reality is ontologicallyh eterogeneous and mixes experiences of individual objects with experiences of models. Models become cognitively useful when they are models of future individuals, and are adequatelya nchored in reality.I f this lastc ondition is satisfied, cognitively advantageous situations mayb eg eneratedw hereby we experience 3D models in the environment of individuals, rather than 2D models in the environment of other 2D models. In this paper, we mainlyc onsider the usefulness of 3D presentations in quite ad ifferent context of possible extensions of knowledge-how. That is, we analyze the possibility that 3D artificial experiences of an agent are sufficientlymultilayered to generate their new manuals kills.T houghs keptical about it,w et ry to formulate conditions which augmented experiencesshould fulfil to generate reliable simulations of new manualskills. We claim that in both areas of knowledge simulation is the crucial component to take into account.W eb elievet hat our cognitive contact with simulations maybeevenbetter than the celebrated contact with reality.Especially,ifsimulationsare able to diminish the risk of failurewhen we strive for useful information or useful manual skill in order to obtain practical goals. If we adhere to the idea that problems should precede pieces of knowledge necessary to solve them, then reliable simulationsm ay by indisputablym orev aluable in solving problems than fullyveridical experience. Thus, we finallycome to aconclusion that AugmentedR eality project givesu sp erhaps as trategic impulset o extend the basic notion of knowledge and ask on what conditions new types of knowledge can be found in the sphere of computer simulations.

SkepticalV Rs cenario
Epistemologists resign themselvest ot he idea that cognitive agents fullyi mmersed in Virtual Reality (VR) cannot have knowledge,p erceptual knowledge, in particular. Let'st ry to explain first who are cognitive agents. Minimally, they are meant to be individuals of sufficient cognitive abilitiesa nd skills to acquire and sustain theirown justified true beliefs. To be an agent requires having cognitive autonomy. Ac riterion of cognitive autonomyi ss elf-knowledge:a n agent has to know who they cognitively are. They need to be cognitively sensitive to their cognitive self. And this seems to involvet he agent'sk nowing what their cognitive skills and capacities are.Another,and more obvious criterion is that to be an agent one to have successful cognitive grasp of reality:a na gent has to know to ad egreew hat reality is like and this knowledge should be acquired in virtue of their owns kills and capacities.I na ny case, reliable evaluation of one'sc ognitive resources is an ecessary condition on having knowledge about reality.¹ The question whether computer simulations could extend one'sk nowledge mayh avea na ffirmative answer if we assume that such simulations( 1) do not violate someone'sc ognitivea gencya nd (2)i nformation that they provide for the agent remains in aw ay reliablyv eridical: it systematicallym aps the reality, or,atleast,i tisnot systematicallycognitivelydeceptive.Here, however,weface adeeper question whether simulations can give us new knowledge.Itseems that Virtual Reality opensthe possibilityofimmediate artificial experience which precedes or even predicts future facts and maybeinaspecial sense veridical. Still, the agent must intellectuallyp rocesst he experience to produce cognitively useful beliefs.The intellectual operation of predictingfuture, which requires special cognitive skills of reasoning, would then changeinto direct seeing of future facts.
We can easilyimagine that the agent is provided with reality-like3Dexperience of an ew type which was impossiblet oa cquire without computer processing.Itsurelyopens awindow of opportunityfor new type of knowledge.Now,we want once again to stress the differenceb etween feeding an agent with experiences and formulating by the agents their ownb eliefs on the basis of these experiences. We will claim thatartificial experiencesneed not to be deceptive if the agent consistentlydistinguishes between objects of reality and models of reality. Along with that, we assumethat ascriptions of cognitive success to someone are fullyj ustified onlyo nt he level of their beliefs, not on the level of experiences. Forsimplicity,weenvisage perceptual experienceshere as connected with naturalc apacities had by an agent,i .e., the senses, whereas the agent'sp erceptual beliefs are connected with something well beyond: skills of interpretation acquired by cognitive training.A gents typicallylearn how to formulate perceptual beliefs and the strength of their agency is measured by their skills of interpretation.
Epistemologists seem equallyfascinated with virtuous and vicious cognitive scenarios involvingc omputer processed information.A" Brain in aV at" (BIV)  These remarks locateour proposal within avastrangeo fv irtue-theoretica pproaches, particularlyt hose of Montmarquet (1987), Sosa (1980Sosa ( ,2 007, 2015 and Zagzebski (1996), where the multilayerednotion of skillful cognitive engagement is central. That said, we will largely remain neutral with respect to other proposals on offer.Where our proposal stands out most clearly, perhaps, is in respect of value-turn aspect of cognitive virtue (see Pritchard2 007).
Is Augmented Reality aS ourceo fN ew Types of Knowledge? scenario is of the latter kind.² Typical variants of BIV are intended to offer situations of full cognitive deception. An important deficiencyo ft hese scenarios, however,isthat they contain explanatory gaps that prevent us from understanding what implies full cognitive deception. In effect,weface the question of what it would takef or aB IV scenario to constituteac redible BIVR scenario − "Brain in Virtual Reality."³ As far as standard BIVsare concerned, epistemologists stress (1) the cognitiveisolation of the discarnate brain, (2)cognitive cruelty of the mad scientistwho manipulates the brain and (3) the unlimited creative capabilities of the computer he uses to feed the brain with artificial experiences.I ti st hen declared thatwhatever the brain is cognitively fed with depends entirelyofthe mad scientistwho controls the computer which is the onlysourceofthe brain'sexperiences.
But consider,very briefly, problems with two variants of BIV: envatmento f cognitiveagency,which is an instance of full deception, and doxastic envatment, whereby beliefs are under threat,e vent hough cognitive agency has not been compromised.⁴ We think that full deception is not an easy task to achievei fo ne assumes that the brain should be deceivedo nlyo nt he level of its beliefs:t he brain would be fullydeceivedifitacquiredfalse beliefs at the manipulator'swill. However,f or it to acquire anyb elief, the brain must retain cognitive agency, which involves the capacity for self-evaluation of its abilities and skills. This requirement mayc ause unpleasant restrictions on the manipulator'sa lleged omnipotence. After all, how does it happen that the discarnate brain remains convinced of his intact cognitive agency?H ow is it made the caset hat although in no way the brain controls its cognitiveprocesses it is stillconvincedthatitdoes?Evenif the manipulator materiallyfaces abrain, he must cognitively challengeanagent. Here, we think, the manipulator has two options to choose from, and both equally mysterious. Firstly, he mayt ry to reconstruct the original cognitivea gency of the brain. How could he know how to do this?How could he have afull access to the original cognitivea gency of the brain?S econdly, he could wash the brain and overlayoni tanew cognitive agency. Again, what would that mean?E ither option leads to the following problem: How is it possible to deceive the brain  Fort he classical formulation see Putnam (2000). Ford istinct interpretations and treatments see Brueckner (1986) and Warfield (1995). Forthe most recent assessment of skeptical strategies including BIV,s ee Pritchard( 2016) and Wright (1992).  Dennett( 1993) outlines an umber of concerns in as imilar spirit.  In arelated contextofcognitive extension Pritchard (2010) maintains that in the BIV scenario an agent could easilybef ed beliefs of various types.Wec ontend that doxastic envatment is not an easy epistemic situation to achieve. about its cognitive agency without recreating its cognitive autonomy? We think it is exceptionallydifficult to state what processes the recreation should involveto be successful. In effect,wesuspect that epistemologists do not quite know what they talk about when they consider cognitive implications of "Brain in aV at" scenarios.
The doxastic envatment variant of the scenario seems ab it simpler.I tc oncerns causal chains resulting in the brain'sa rtificial experiences. Specifically, causal chains resulting in perceptual experience of the brain are always mediated by the computer.F or instance,they maybegin within the computer and then lead to deceptive experience,o rb egin within the reality,b ut then gett ransformedb yt he computer in ways that make experience unreliable. Crucially, and this presumably is the most speculative component of the story,d eception comingf rom the scientist via the computer mainlyc onsists in their switching at will between falsidical and veridical perceptual experiences to provoke the brain − made to believetosome extent that it is acognitive agent − to formulate false beliefs about the reality.Whether beliefs turn out true or false does not matter much, since falsidical experience thatunderpins them is ex hypothesi totally worthless, whereas veridical experience is unreliable and,therefore, hardlyb etter.H erea gain, however,o ne faces the question: How to engineer av irtual environment (like ar eal one!) that supportst he generation or maintenanceo f false beliefs about reality withoutt he brain − which is to as ignificant degree cognitively autonomous − ever forming abelief that its experience is deceptive?
There are no proposals to explain whyt he brain believes thati ti sa na gent having typical cognitive abilities and special cognitive character;how the manipulator makes the brain think that it has aliving bodyw hich is perceptuallysensitive and skillful in acting,h as its own cognitive aims, own practical targets, and so on.
It is not our aim, however,toengagewith BIV scenarios in detail, but merely to highlight thatthey purport to show,one-sidedly, that Virtual Reality is cognitivelyu seless without explaining why. Usuallyt hey are imprecise as to whether and how the brain retains its cognitive agency, how it is possibletoproduce for it af alse cognitive agency, how the false cognitive agency is to be fed with experience to make the brain cognitivelys atisfied, and so on. What matters for our purposes is that radical scenarios end with the pessimistic conclusion, not necessarilywelljustified, that no VR information in no VR world for no VR agent can become af orm of knowledge.

Benign scenarios involving computer information
Normally, we take it for granted when using computers thatour cognitiveagency is impregnable to as ufficient degree. To be sure, the quality of interaction with computers is becomingincreasinglyagent-like. Nevertheless, as thingsstand we give our computers no chance to deceive us as to who we are as cognitive agents, and they are frequentlyreduced to being our sources of information. Crucially, if we see ac omputer as an independent external object with some information written on its screen, there is oftenn op roblem with ascribing al ogical value to the information and to decide whether it is reliable enough to provide for us as ourceo fk nowledge-that.S imilarly,i fw ew atch 2D picturesd isplayedo n ac omputer screen which is as eparate object within our field of view among other external objects, it is often easy enough to decidew hether the computer pictures carry true information, e. g., whether they map the reality around us in epistemicallysalient ways.Whenever that'sthe case, the basic epistemic problem is just that: What cognitive use are we able to make of the information?

Operational knowledge-that
Let'sd istinguishb etween factive knowledge-that,a nd operational knowledgethat.F or example, as as killed car driverE la believes that GPS trulys hows to her bothwheres he is and wheres he should go.The first information is factive, whereas the second is operational. If GPS reliablyt ransmits true information, there is no problem with ascribing to her knowledge that she is here-and-so and that she should go there-and-so. Both typeso fk nowledge are interconnected: GPS enables Ela to know whereshe should drive because it also enables her know wheres he is. Typically, GPS knowledge is of essential instrumental value, i. e., often one wantst ok now whereo ne is not because they justw ant to know wherethey are but because it helps them to move efficiently; and,inEla'scase, to reach some previouslye stablished destination point. If Ela is as killful interpreter of GPS visualizations, these help her to remain on the right track towardsher final destination. Arguably,this type of knowledge is of an ew type because she knows wheres he is and wheret og oi nt he surroundings that are totallyunfamiliar to her.Factively,she maybelost,but operationally she is still in control. With this kind of operational knowledge she can movee fficiently.I nc ontrast,i fs he onlyh ad factive knowledge wheres he is and what destination point she wants to reach, this knowledge in unknown surroundings would be useless.
Locally, GPS information maybefunctionallyonapar with information provided by as ystem of road signs which are all elements of reality.F or instance, some signs point Ela in the right direction, so when she sees and follows these, she doesn'tneed to follow informationfrom GPS. Indeed, GPS has no monopolyonguiding her driving,though usuallyitguides her more efficientlyand, absent technological mishaps,m ore reliablyt hanr oad signs.B ut there are important differences in what the road signs and GPS afford to Ela. First,u nlike signs,G PS makes her driving in unknown areas as smootha sh er driving in well-known areas.B ut therei sa lso ar obust qualitative difference that goes beyond the degree of efficiency,n amely,that GPS opens for Ela an ew perspective on moving.Usingit, Ela mayidentify her position and away to adestination in unknown surroundings: starting in any unknown place Ela maymoveefficiently to anyother unknown place. She can navigate without first becoming anavigator herself. Now,wefind it vital for our discussion of Augmented Reality to introduce the following enablingc ondition. When digital informationi sdisplayedi ns uch a wayt hat the computer and its screen are invisible to you, and it seems to you that the information is placed immediatelyi nreality which surrounds you, then youfind yourself in AugmentedReality.Wethink an important definitional implication is that when computer-generated informationtransforms Plain Reality into Augmented Reality,the information enriches your overall experience in a mode that bypasses and, is experientiallyinsulatedfrom, experiences caused by external objects. And if we could preciselydefine what are the new typeso fe xperiences, then we could alsodefine new types of experiential knowledge.Aswe see it,then, GPS projections on one'sm obile phone screen are not yetapart of Augmented Reality,t hough they easilyc ould be. To satisfy the condition, it is enough to makei nvisible the screen on which the information is primarily displayed.
Consider the following to be as omewhat more precise elaboration of that condition. So long as acomputer screen is avisibleexternal object,and together with other external objects co-creates one'sf ield of view,then what one experiences is treated as part of Plain Reality.Plain Reality transforms into Augmented Reality when one sees information that is displayedonafullytransparent screen which creates new boundaries of one'sf ield of view,b ut is not itself ap art of their field of view.I fo ne has the impression that anyp ictorial or written informationt hey see is present immediatelya mong external objects, then they face what is called Augmented Reality.F inally, one finds oneself within MixedR eality,ifwhen equipped with adevice containing atransparent screen, theycan ex-perience 3D objectswhich seem to be placed in the fragment of reality that they actuallysee through the screen. The simplest wayofm aking GPS aparto fA ugmented Reality is by generating av isible secondary screen within an invisible primary screen. These are basic tricks of Mixed Reality:t og enerate 3D artificial experience by displaying information on at ransparent screen or by displaying new 2D visible secondary screens behind the invisible primary screen.
There are twos ystematic illusions connected with creating an Augmented Reality.F irstly,e veni fascreen is necessary to generate projections,i tm ust be so positioned as if there weren os creen at all, that is, as if youh ad an unmediated visual contact onlywith external objects and via unaided eye. Secondly, all 3D projections are illusory in the following sense: Reality remains unchanged, onlyo ur artificial experiences change. Aq uestion to answer is what can be the cognitive status of 3D projections.Ifthey are virtual, how could they be sources of knowledge-that?

Knowledge-how to act
Computer information mayb ed ecisive in acquiringk nowledge-how to act.I n particular, simulatorsw hen coupled with computer simulations seem highlyeffective in supporting the acquisition of manualk nowledge-how to act.F or instance, we can significantlydeepen our skills of driving ac ar.Technically, standard simulators involvereal tools,such as adriving wheel to manipulate, along with computer simulations projected on ac omputer screen which visualize predicted results of our manipulations. What is important here is that standard simulationp rocedures combine real causes, real actions, and predicted unreal results,with our bodies being engaged in manipulating material tools essentially similar to those we will use in performingr eal actions. Forexample, we manipulate awheel which functionallyresembles areal driving wheel with the expected final result being ar eal skill acquired by simulated training.S tandards imulators are valuable because they enable us to learn through "trial and error" with no negative results characteristic of real errors:when we make simulated errors no real damagei sd one. We can learn on errors which have onlyv irtual consequences.
In aspecial sense, such knowledge-how is of an ew type because virtual errors are pedagogicallya se fficient as real errors. At this juncture, an important question arisesw hether we can go anyf urther in learning by computer simulations and applyAugmented Reality to acquire real manual skills either via (1) reversing the order of causes and results, i. e., starting with simulated causes and ending with real effects, or (2)t otallyb reakingt he pattern by substitutingr eal causes and effects with simulated caused and effects.
Arguably, when we entirelyseparate ourselvesfrom reality,weare unable to acquirem anuals kills. Fore xample, we cannot learn how to drive ac ar onlyb y making thoughte xperiments,i .e., by imaginingt hat we manipulate ad riving wheel in ac ertain waya nd imaginingt hat the results we obtain are such and such. An argument supportingthis claim would be that these imaginary simulations cannot work because there is no reliable connection between imaginary moving of an imaginary driving wheel and imaginary results of imaginary driving.These imaginary causesa nd the results are in no waya nchored in reality. Now,aquestion mayb ea sked whether we can eliminate the drawbacks of thought experiments by simulations made within Mixed Reality.Whether we can learn how to drive ac ar by applying av irtual wheel of av irtual car in a real scenery or av irtual wheel connected with av irtual secondary screen presenting virtual surroundings.I ts eems thatm uch would depend on anchoring all the important elements of such augmented simulation in reality.B ut then still further problems arise. To acquire am anuals kill something more is necessary than new visual experiences.Wen eed new bodilyexperiencess uch as,f or example, new kinaesthetic feelings. Such feelings do not arise without tactile experiences in our body. But it is unclear whether Magic Leapproject,for example, has ambitions to equip us with new artificial tactile experiences.⁵

Non-standard simulations and new knowledge-how
People alone cannot transform stored informationa bout theirp ast natural experiences into the sameexperiences again: they cannot re-vitalize past experiences.They alsocannot see objectsthey onlyhavethought about or imagined: they cannot pre-vitalize futureexperiences. In contrast,information stored in computer'smemory can be repeatedlytransformed into people'sartificial perceptual experience. This opens the doors for experiences transcendingtime.Similarly, people cannot generate visual simulations of objects, events or actions at will, but computers give them this special possibility.
 Here one maya lso find it useful to engagem orec loselyw ith distributed cognition frameworks, particularlythose that focus on embodied and enactive character of skill, to then present ac ase for an ew type of tactile cognitive extension through AR.
Is Augmented Reality aS ourceo fN ew Types of Knowledge?
In some sense standard characterisations of AugmentedR eality,f ollowing Azuma's( 1997) definition which requires thati tc ombine the real and virtual, be interactivei nr eal time and registered in 3D,a re somewhat misleading since they mayincite the hope thatcomputer technologies are aimed at bringing direct changes in the world. It is obvious that augmentation makes sophisticated hardware appear in the world, but when we speak of the resultsofaugmentation it should be clear that we meansolelychanges in our experience caused by this hardware. Firstlyt hen, to straighten out the matter of what augmentation may change, it is useful to stress that (1) experience alone does not changet he world and (2)the world provides the ultimatepoint of reference, but in decidedly different ways,both for our natural experiencesand artificial experiencesgenerated by computer technologies. Natural experiences are caused by,whereas artificial experiencesm ust be registered with, the world to count for us. The world does not materiallyc hangeb ya ugmentation though our experience may changeimmensely, and we augment our experience reasonablywhen, for example, inspired by instrumental reasoninga bout means to obtain ends, we decide to make ac hangei nt he world and, before we actuallym ake it,thanks to artificial experience we feel well prepared to succeed in making it.W ed eliberately changethe world onlybyactions, although we mayprecede the actions with deliberate augmentations.
Thus, the issue of Augmented Reality begins to look serious if we manipulate our experiencesm otivated by problems we face and our perseverance to solve them. Anyway,h ere we distinguish between capricious augmentations which are art for its owns ake and purposeful augmentations which prepareu st ou ndertake actions to successfullychangethe world in order to improveour position in it.
From the point of view of epistemology,inspiring are these definitional conditionsw hich requiret hat Augmented Reality be am edium for displaying 3D presentations (1) registered with the world and (2)p henomenallya djusted to the world and the agent who perceivesthem.⁶ These conditions imply, for example, thatifsome places are choseninthe world as stable points of reference and an agent using Augmented Reality glasses is amobile point of reference, then the wayh ei se xperiencing virtual objects is fullyc onsistent with the wayh ei se xperiencing material objects. Experiences of both types should follow the same rules of presentation, or their basic phenomenologyshould have the same gram- Foracharacterization of AR as medium rather than at ype of technology itself see Craig (2013). Forasurvey of definitions see Billinghurst et al. (2015) and Grubert et al. (2016). The latter discuss conditions for pervasive Augmented Reality that would offer continuous experienceo f reality-registeredp resentations and be context-aware. mar.Thatis, if the shape avirtual object is experienced in aparticularway by an agent,the shape of its physical counterpart would be experienced by him in the same way.
Ad esirable property of augmented experiences is theirp lasticity.N atural visual experiences are not essentiallyp lastic in relation to the agent.T or etain veridicality,their content must be determined by external causes, i. e., properties of material objects. Augmented Reality offers the agent an option of controlling the substance of their artificial experiences. And it is fascinating that artificial experience maybecome interactive in relation to its agents, whereas natural experience is interactive mainlyinrelation to its objects. The property of plasticity is intellectuallyp romisingw hen manipulations are meant to display, for example, as equence of possible states of affairs to allow us to make the best choice between them.
As eparate question is whether the plasticity of experience is practically promising, i. e., whether by manipulating artificial experiences we can improve our manual skills; whether,i np articular,wec an acquiren ew manual skills by manipulating our experiencesofmanipulatingvirtual objects, instead of manipulating material objects. It is important whether Augmented Reality technologies can simulate causal connections between, for example, artificial experiences of an agent and bodilys kills of the agent.S tandards imulators destinedf or teaching manual skills connect authentic bodilyactions engagingp hysical toolswith computer simulations of their results in some virtual surroundings.That is, such simulations predict onlyt he results of actually performed actions: real causes are combined with reliablyp redicted though unreal effects.
Could we introduce non-standard simulatorswhich would reverse the order of real causes and simulated effects while retaining the reliability of standard simulators (i. e. ones wherew eb egin with virtual causesa nd reliablye nd with real effects)? Think, for instance, of ag olf strokes imulator that mixes strokes performedw ith real clubs and balls with virtual trajectories of the balls visualized on as creen. Can it allow for the possibility of augmented strokes? The person in such as cenario sees their bodyp erforming moves characteristic of a stroke, but the strokei sp artiallya ugmented as it involves artificial experience of ac lub. They strike the ball by engaging their real body, in the real surroundingsofagolf course, yetits resultisartificiallyvisualized in the real space of the course. Couldwemake such reverse simulations reliable? If we could, we could also improveo ur game by such simulations. This would mean that we have an unquestionablyn ew sourceo fk nowledge-how to act.
Thus, we face manyquestions concerning the possibilityofextension both of the methodologyoflearning how to act and, further,the environment of acting; the possibility that our actions easilyswitchbetween Plain Reality and Augment-ed Reality.The fundamental one is whether it is practicallyi lluminatingt om ix manualmoves of abodywith virtual tools and observetheir virtual results in the real world. We show below how one could learn intellectuallyb ym ixing two worlds of objects, but the question remains whether one can alsolearn manually by mixing twoworlds of actions. Especially, when the learning begins with highly augmented bodilym oves and but its aim are real results of the moves.
It seems uncontroversialthata ugmentation maydevelop the intellectual aspect of manualactions because spatio-temporal manipulations mayturn out sufficientlyh elpful to decide whether or not one encounters an opportunity to act. We can plan virtuallyhow to make the best use of the skills we alreadypossess. But it is uncertain whether augmentation givesusachance to improvethe skills themselves, or whether it always makes sense to try to improvethem by augmentation.
Let'stake acloserlook at the golf shot case. When one has artificial experience of agolf club,then its location is independent of where one is and how they move. One clearlyisthe viewer of the virtual club. But can they also be the user of the virtual club?I fo ne'sa rtificial experience of ac lub werem ixed with their natural experience of ag olfc ourse, then the experience of the club would be rooted in ahardware, whereas the experience of the golf course would be rooted in reality.T here would, then, be two independent causal chains of their intertwined experience, one thath ingeso nt he external world and the othert hat is enabled by computer stored information. Generally, wherevirtual objects are involved, these exist imprisoned in ac omputer,a sopposed to the physical world. Optimists about artificial experience mayt ry to persuade us that due to its spatio-temporal registration with the world, the club mayb ef elt present in the world.⁷ But spatio-temporal location is onlyt he tip of the icebergw hen we try to understand what would be the profile of artificial experience such as to make one feel, and believe, that they are actuallyp roducing ag olf shot with a virtual club and ball. Minimally, ar eliable simulation seems requiredo fo ne's bodilycontact with the ball via the club, areliable simulation of the physical impact of the club on the ball and, finally,areliable simulation of the resultofthe impact.Ifthis can be done, Augmented Reality will become apowerful sourceof knowledge-how.But can this be done? And, further,does it always make sense to produce non-standard simulations?
Apart from its location in the world, ap hysical club also has dispositional properties that determine its identity as as pecific tool. It is in virtue of these properties that the club respondsi nc ertain ways to certain bodilya ctions of  See Craig (2013). the playerand can be used purposefully. If we are not mistaken, Augmented Reality technologies at their present stageofdevelopment enable one to changethe location of avirtual club, however,iti sd ubious whether they also enable them to use it.
Clearly, one cannot directlys trike ap hysical ballw ith av irtual club,i .e., with their artificial experience of ac lub. And this is because there are no direct causal links between one'sb ody, their experienceso fv irtual objects such as a club, and the targetp hysical object − in our case, the ball. Now,d irect causal links would be dispensable if they could be reliablysimulated. Could the causal links between one'sb odya nd ap hysical object,which standardlyc reate an action, be reliablysimulated when, with theirbody, one intends to make some use of that object'sv irtual counterpart?T his troublesome question demands ap recise answer.And as long as we don'tknow what types of dispositional properties of virtual objectsc ould be reliablys imulated, we also don'tk now what actions engagingt hese objects could be simulated.
At present,r eal objects and virtual objects have very distant grammars of dispositional properties.That is, our interactions with virtual objectsa re so far one-sided and primitive in comparison with interactions we can have with physical objectsinPlain Reality.And it is hard to predict whether this distance willbe essentiallys hortened and which dispositionalp roperties will be selected as worth simulating.A nyway, as we need in advancealist of dispositional properties of physical objects if we want to evaluatet heir manualu tility, similarly, we will need to have al ist of respective simulations if we want to act in Mixed Reality effectively and reasonably.
There are,therefore, two limitations to break if we are to acquirenew knowledge-how while beingimmersed in Mixed Reality.First limitation pertains to the physical impact thatone'sbodycan be simulated to have on virtual objects. The second one concerns the impact that virtual objectscan be simulated to have on the physical world, includingone'sbody. So far,itisunclear whether the breaking has been started. Though spatio-temporal registration with the world is undoubtedlyasuccess, something we could call "dispositional registration" would surelyoutbalance its significance.I twould be quitestimulating if we could manipulate dispositionalp roperties of virtual objects, eliminating the undesirable and adding or enhancing the desirable ones. But,a gain, we don'tk now when such augmentations will be possible, and so it is difficult to say, for example, which types of simulations we should avoid or accept.

Chances of new types of knowledge-that
It is said that some 3D virtual objects mayhavetheir physical counterparts. This claim is somewhat disputable. If one entertains 3D artificial experience of ag olf club which is modelled on ap articularc lub that one actuallyh as, is their real club acounterpart of the virtual club?Isthe virtual club one sees an individual? Generally, are 3D virtual objects one sees via Augmented Reality headsets individuals?T here are reasons to hold they are not.
Firstly, the objectso fa rtificial experience functiona s, and have properties of, models, i. e., general objects. They are the samef or all users of ah eadset. i. e., the same digital informationi sp rocessed to produce the same artificial experiences for manyp eople. When they are displayed, all users of headsets of a type enjoy qualitatively the same possibilities of experiences.Objects displayed in Mixed Reality presentations provide, of course, various sorts of models: (1) Models of objectsw hich exist,( 2) models of non-existing objectsw hich are intended to exist and, finally, models of the unreal objectswhich are not intended to exist.A nd if we weret obet which of them are the right candidates for vehicles of new types of knowledge-that,o ur choice would be that models of intended objects.
By models we mean objectsthat are neither autonomous nor unique entities: objects that can be perfectly replicated, or repeatedlydisplayedbymanydisplayers, with no changei nt heir visible properties. Similarly,i fm odels undergoa structural change, the changei sd etermined or initiated externally: they are not sources of theirown structural changes and, especially, they do not autonomouslye volve. Ac rucial property of 3D models is that they cannot be changed by direct external impact comingfrom real objects or agents; especially, from direct impact initiated by bodilya ctions of human agents.
What more can we gett ok now from Mixed Reality that we cannot gett o know from Plain Reality alone? If we ask this question thereisapreliminary demarcation line to draw between projections made for mere entertainment,orfor purelym ercantile purposes, and thosem ade for serious cognitive purposes.⁸ To the extent that computer games are usually made to entertain the consumer,and for financial gain of the seller, we will largely set aside the possibilities of their development.O fc ourse some games escape this qualification, especiallyw hen they directlys ervee ducational purposes. In anyc ase, we expect thati fs erious  Foraroughlys imilar point on models see Hofmann (2013) whoh olds that ad istinguishing featureo f" serious" simulations is their validity,w hereb asic assumptions about whate xists, how to come to know,a nd how to achieveagoal are keyd eterminingf actors at play. augmentations of reality entail essentialo ntologicale xtensions of reality,t hey should bring original additions to people'sb odies of knowledge.I fw el ook through advertisements of Microsoft or Magic Leapt echnologies, it is often unclear whether they entail such extensions or in what sense they do.⁹

Seeing models in real surroundings
Mixed Reality (MR) presentations seem sometimes cognitively idle in that they are aimedt om ake am omentary impression: their programmedr esult is "Wow!" of the viewer; an exclamation of astonishment.A dvertisements of MR equipment offer al ot of cognitively undetermined simulations. That is, when we watch them it is difficult to decide whether they carry anyc lear message for the viewer as to how to consume them intellectuallyo rp ractically. As if the ambition of these technologies was restricted onlytowhether artificial experiences alreadya re, or are not yet, qualitatively indistinguishable from natural experiences; as if nothing more should be required of them beyond the mere capacity to "fool our senses".F ortunately,M Rl eaders unanimouslyd eclare their technologies are intended not to fool us as cognitive agents but,onthe contrary, to help us cognitively.H ow can they help?
One promisingdevelopment consists in changingt he wayofd esigningn ew objects which are intended to exist.Where the geometry of an object is essential for its future functioning − no matterofw hat type the object is to be − making spatial models of such objects is indispensable before producing real individuals. The old-fashioned method of designingt he geometry of am odel consisted in drawing on paper three2 Dp rojections of its intended shape. After which a key turning point came: movable 3D computer projections on avisible computer screen. With recent advancements the question has become: what can we gain when we completeour designs with 3D augmentations, i. e., whenweexperience a3Dfinal model of an object in some real surroundings?The answer is that the closer to the reality the model is, the better we know whether the real object will fit the reality.Obviously, we get "perfect" knowledge about fitting onlyw hen we see how areal object fits real surroundings.But the second best optionistosee a 3D model in real surroundings. Designers acknowledge that the environment of an object is important.For example, architects aim to anticipatewhether ahouse will fit certain real surroundings geometricallyand aesthetically; and it is not uncommon that the view of the real object in its real surroundingsdoes not satisfy  Fora ni nformative discussiono ft he enigmatic Magic Leap see Kelly( 2016).
Is Augmented Reality aS ourceo fN ew Types of Knowledge? the expectations of its designer;p articularly, the real object sometimes looks worse in real surroundings thanits model looked in fullyartificial surroundings. Although the profit of MR designingi ss ometimes minor − as when the model shows the shape but cannot show the functioning of the futureo bject − we undoubtedlygain anew type of knowledge when what we see now is as close to the future as possible.
Importantly, the value of such knowledge-that increases if the object and its surroundings form aw hole. If in planning an ew type of object we must also plan its physical and functional contexts of functioning,a nd when its functioning tightlyd epends on other objects' functioning,t hen it makes aq ualitative epistemic differencet hat we oppose 3D projections and real objects. Ar eason is that reality is coarse-grained, whereas models are ideal, or finely-grained. It is an unpleasant shock when it turns out that am odel embedded in another model fitted perfectlyb ut ar eal object embedded in real surroundingsd oes not.Typically, old-fashioned designs consisted of models within models: of models of objects embedded in models of their surroundings.F rom that angle, transition to the model-reality pair is surelya dvantageous as it makes designing more reliable and more flexible. We mayc hoose what to put into what.S ometimes it is preferable to put ar eal object into artificial surroundings, sometimes it is preferable to put an artificial object into real surroundings.

Seeing the inner structure
An identification mark of the old-fashioned designing were2 Dc ross-sections of models. As 2D projections showed usually the exterior planes of an object,a lways when it was also necessary to revealk ey connections of its interior elements, 2D cross-sections wereu nbeatable. They were necessary to show that the design was internallyconsistent,that there weren oconflicts between its elements. As ah ouse, for example, includes different installations, wiring,ventilation,g as fittings, among others, and they must be skillfullyc oordinated, cross-sections weren ecessary to show that they were. Now,3 Dm odels offer the possibilityo fs eeing the full interior of ad esigned object in anys cale. We can at will see the interior of af uture housei ni ts full scale, along with every element in its real dimensions. It is, therefore, temptingtoclaim that we can perceive now afuturehouse, or even that we now visuallyknow the house although it does not yetexist.Ofcourse, there is adifferencebetween seeing amodel and seeing areal house. Alot mayhappen between having afinished model and having at argetr eal object.B ut sometimes we are certain that the difference will be negligible. Although the model and object mayeventuallydiffer,this will not be because the design was internallydefective.A nd this is, we think,the main criterion of having knowledge-that: that one is in possession of information which essentiallyd iminishes the risk of theiri ntellectual or practical failure.
We fail practically, for example, when we cannot predict that an object is mechanicallyu nsound,i .e., is not readyt of ulfil its standard function. Hencew e must not avoid the question whether experimenting with the geometry of an object is essentiallyhelpful for predictingf uturemechanics of the object.D oes artificial experience help to predict whether,f or example, ag eometricallys ound model is alsom echanicallys ound?I tm ay to ad egree, if we know in advance the material properties of its elements and we do not need to experiment with their mechanical adequacy. Nevertheless, evaluatingt he mechanicals oundness of afuture object is far more complicated than evaluatingits geometricalsoundness. At that point,wei nevitablyr eturn to the question about seeing the future which from the beginning was of special significance: Can MR projections show us also the future mechanics of the designed object,show its futureinner mechanism at work, especially, if its work is to be dynamic, involving not onlymoves of many elements but also theirfunctional interaction. If testingamechanism is necessary,modeling usually becomes an insufficient endeavorbecause sooneror later we need something more than amodel, namely, aprototype of the designed object.Inthis way, we come to the question whether MR technologies mayequip us with models having characteristics of prototypes, with models, for example, enablingu st ot est the future work of an intended object;t oh elp us to decide that the model is mechanicallys ound. It would surelyb ea ne pistemic breakthrough if we could now know how an object will work because we can see now ar eliable MR simulation of its moves. But we doubtt hat seeing moveso f amodel makes us know the future work of ar eal object.Ifweknow in advance how ar eal object works,M Rp resentations mayb es od esigned as to show us how it works.But if we don'tknow in advancehow an object works,MRmodeling of its movesw ill not reliablyp redict its workings.Thus we need to sharply distinguish between virtual presentations of the mechanics of known objects and virtual tests of the mechanics of newlyd esigned objects. Virtual presentations mayb ep edagogicallyu seful, but are useless when we need reliable tests to acquire knowledge of an ew type. We are uncertain whether MR technologies aim at all at producing simulationsofreliable prototypes or reliable testing. But as long as they don't, their presentations are mechanicallyidle in the sense that although they explain al ot of what we alreadyk now,t hey predictv ery little. If they don'tlet us test an unreal object functionally, no new window of epistemic opportunity opens.

The instrumental value of knowledge
It is hard to decide whetherM Re xperiences ares ources of newk nowledge until oneestablishes what to expect from ourbeliefs.Our view is that beliefs, andperceptualbeliefs in particular,provide toolswhich should be utilized,i ntellectually or practically.A nd although we accept thedistinction between( 1) autotelicv alue of knowledge-that − consisting mainly in itstruthfulness − andits (2)instrumental value, we don'tc are much aboutt he former.T oo ur mind,beliefs become instrumentally valuable if we efficientlyp rocess them intellectually to gain ar esult: afinal belief whichbringsasolution to an intellectual problem.Ifbeliefs become reliably efficientinthe serviceofsolving problems,theycan be classified as pieces of knowledge-that. As we seeit, efficiency in solvingproblemsisaninterestingcriterion of possessing knowledge: beliefswehaveare pieces of knowledge if they at leastd iminisho ur failures in achieving goals. Fore xample,wem ake an intellectual useo ff actualb eliefs whenw eu se them in explanationso fp astf acts, or in making predictionso ff uturef acts.I ft he explanations/predictions work,b eliefs that contribute to them become pieces of knowledge-that. More importantly,p erhaps,w ev iewa sk nowledge also thoseb eliefs that efficientlys upport solutions to ourp ractical problems;b eliefs that help us to decide whethert oa ct or, whichisevenmorecrucial,how to act. Shortly, knowledge is what equips itspossessorswithinformation that is necessaryfor them notonlytothinkrationallybut also to actr ationallya nd,wes houlds tress, skillfully.

Conclusions: The scope of artificial knowledge
Thei ssue we investigatedi nt hisp aper -i.e.,whether 3D presentationsm ay initiaten ew typeso fk nowledge − provides us with problems olving information whichcould be generally labelled "artificial knowledge".Ifweare epistemological conservativeswho hold fast to ther equirement that knowledge must mean "epistemiccontact with reality",thenthe answer will be negative.Obviously,suchconclusioni mplies that AugmentedR eality presentationsa re nots ources of knowledge-that. If,h owever,w es witcht oi nstrumentalism, reasonsq uickly emerge to startspeakingabout perceptual knowledge-that aboutthe future;especially, if artificial experience presents us with models of objectsw hich we have alreadyd ecidedtomaterialise andthe modelsare (1)sufficientlyclose to realityand (2)reinforceour positionsasagents. It is of course atrickyquestionwhether,for example, oneseesn ow af uture housew hent heysee nowa3D modelo fahouse. We suggest that underaspecialconditionthe answer should be positive:whenthe model significantlydiminishesthe risk of failurewhenwefinally switch to acting in Plain Reality. If,for example, thesoundness of themodel hadbeenreliablytestedbefore we turned to producing ar ealo bjecto rp erform ar eala ction.
MR visualizations do not bring knowledge-that of an ew type unless they open for us the future in an ew way, i. e., unless the visualized models are registered with reality.Westressed throughout the paper that the registration should take avariety of forms if the rangeofnew knowledge is to qualitatively widen. At the same time, it remains unclear to us which forms of registration could be actualised and, importantly, whether it would make sense to actualisee very one that could be.
Even once stronginstrumentalism has been adopted, we maystill find it difficult to decidethe question. After all, does one have perceptual knowledge-that when they see pure MR presentation, presentation being no test of soundness?If MR presentations which tested nothing weren ot cases of perception, we would then have strangec ases of handicapped visual experience which cannot turn into perception. Still, we mainlyasked with what properties of models we should experiment to profit from artificial experiencesa nd we suggested that experiences testinggeometrical soundness are inferior in relation to the ones testing mechanicals oundness.
We admit thatour ultimatequestion was whether Augmented Reality allows us to experiment with our agentive soundness, i. e., to test the soundness of our manualskills. Herewedid not movebeyond speculations. On the one hand,itis obvious to us thatreliable simulationisanextraordinary tool for making manual progress.And that it would be superb to have artificial experiences which could createt he opportunity of reliable simulation. Givent hat,wef ind it quite stimulating to ask questions such as: Can we experiment with manualuse of VR models placed in real surroundings,i na ne ssentiallys imilar wayt ot he wayw ee xperiment with real objects placed in real surroundings,w henw em ake plain experiments, or at least with real objects placed in unreal surroundings,a s when we use standard simulators?Itremains unclear to us whether MR technologies creates uch opportunities.
Although propagandists speak of unlimited capabilities of MR technologies, we find the issue of developing our manual skills within MR uncomfortable for them. If we lack reliable simulations of direct bodilycontact with VR-objects, if we do not learn from simulationsoftouchingthem, if we do not make anysimulated impact on them with our bare hands, then our knowledge-how cannot progress in an ew way. Additionally, there are so manyp ossible mutations of manualactions' simulations within MR thati tb ecomes reallydifficult to decide which mutations are still challengingand which are alreadyabsurd. If, for example, am anual action involves an agent,s ome toolsa nd some surroundings,i n