Intelligent Practical Reasoning for Autonomous Agents: An Introduction

This paper is an introduction to recent work on practical (means-end, goal-directed) reasoning in artificial intelligence. By using an example of community deliberation concerning whether to change to a no-fault system of insurance, it is explained how practical reasoning is used in public deliberation. It is shown how argument mapping and argumentation schemes are useful tools for modeling the structure of the argumentation in such cases. The distinction between instrumental practical reasoning and value-based practical reasoning is modeled using argumentation schemes.

Practical reasoning, also called means-end reasoning or goal-based reasoning to action, is modeled in current studies in artificial intelligence (AI) as goal-directed reasoning used by an autonomous agent to conclude to a decision to act, based on its goals and its knowledge of its circumstances. Goal-based reasoning of this sort is very important in areas such as AI, multi-agent systems, cognitive systems and robotics. This paper surveys recent research that builds formal and computational models of this kind of reasoning based on an argumentation approach that takes into account how agents communicate with each other to work on a collective task. The paper shows how practical reasoning is employed in settings of rational deliberation where intelligent agents try to collectively arrive at a conclusion on what they should do to move forward in a given set of circumstances. The paper explains how recent work has applied argumentation models to help build more realistic computational systems of deliberation, such as those that might be helpful in deliberative democracy.
In the first section it is shown how practical reasoning is used as a form of argument in a typical case of defeasible reasoning from a set of premises to a conclusion to take action. The main objectives of this section are to introduce the basic argumentation schemes for practical reasoning and to show how critical questions and counter-arguments can be used to evaluate instances of practical reasoning in particular cases. In the second section it is shown that the entity that carries out practical reasoning can be defined as an intelligent autonomous agent where the term "intelligent" means that such an agent reasons not only from its goals, and from what it takes to be the means required to attain these goals, but also on the basis of the factual circumstances of its given situation. The third section surveys the literature comprised of different approaches to practical reasoning. The fourth section uses a real example of community decision-making about whether to change to a no-fault system of insurance to show how practical reasoning is commonly used as a species of argumentation in typical political deliberations. In sections 5, 6 and 7, the differences between purely instrumental practical reasoning and value-based practical reasoning are explained. In section 8 it is briefly explained how practical reasoning is used in computational models of deliberation useful for multiagent systems. It is shown how research on practical reasoning can move forward by combining abstract models of argumentation in deliberation developed in AI with practical studies that need to apply these models to argumentation in natural language discourse. To show how practical reasoning depends on the capability of an intelligent agent to judge information about its factual circumstances, section 9 distinguishes between two goal-directed communicative frameworks called deliberation dialogue and persuasion dialogue. Section 10 draws some conclusions and provides an overview of how the flow of practical reasoning works as a connected sequence of argumentation in such settings.

Heuristic Practical Reasoning
The following scheme is a heuristic form of what is called practical reasoning in the philosophy literature and goal-based reasoning in computer science. A heuristic is a shortcut that leads temporarily to a conclusion that may later have to be revised. Stripped to its basics, the heuristic form of practical reasoning can be represented by the argumentation scheme below, with its two premises and single conclusion (Walton, Reed, & Macagno, 2008, p. 323). The "I" in this scheme stands for an autonomous intelligent agent (IAA) of a kind fully defined in section 2. Minimally speaking, an IAA is an entity that has the capability of forming goals and the capability of carrying out actions that it thinks might realize one or more of its goals. An agent in this sense can be machine or a human (or an animal).
Goal Premise: I have a goal, G.
Means Premise: Carrying out this action A is a means to realize G.
Conclusion: I ought (practically speaking) to carry out this action A. Practical reasoning is extremely common. We use it all the time in our daily human activities. The heuristic form of an argument is a stripped-down version of the argument that jumps from the premises to the conclusion in a rapid way that can make it tentatively useful, but that can also overlook important counter considerations, such as additional premises that should also be considered or critical questions that might suggest significant reservations about the applicability of the argument to special circumstances of the case at issue. Using the heuristic version of the argument from practical reasoning, an agent jumps quickly from a goal, along with seeing some means to achieve the goal, to an immediate action to carry out the means.
For example, consider a case of a person who has a sleep disorder and is considering taking a certain medication in order to help him sleep peacefully at night. He thinks that this medication will enable him to achieve his goal. The sequence of practical reasoning is displayed visually below in an argument diagram where the rectangular nodes contain propositions and the round nodes contain pro or con arguments. The conclusion of the argument is shown on the left in figure 1. It is the proposition stating that the agent should take the medication. The argumentation scheme for practical reasoning is shown in the argument node to the right of the text box containing the conclusion, indicated as PR. The plus mark inside the node indicates that it is a pro argument. Examining the next two text boxes to the right of the argument node we see that they fit the goal premise and the means premise. The two additional pro arguments shown at the right support the means premise. It can be easily seen using this example how the heuristic form of the scheme for practical reasoning represents a defeasible form of argument that should only be accepted tentatively and should be subject to critical questioning. The following set of critical questions matches the heuristic scheme for practical reasoning (Walton, Reed, & Macagno, 2008, p. 323 In the MedX example, one of the most important critical questions concerns the potential side effects of taking this medication. The appropriate critical question here is (CQ 5 ).
One of the developments in the field of AI that has enhanced our capability to model practical reasoning as a www.ccsenet.org/res Review of European Studies Vol. 8, No. 1;2016 form of argumentation that takes place in groups of IAA's is the availability of more complex models of the argumentation scheme that go beyond a simple heuristic model. The method adopted by the CAS to overcome this limitation is to build the critical questions into the argumentation scheme itself, making the scheme more complex, but also much more powerful and useful as applied to realistic examples of practical reasoning. These developments were precipitated by the observation that it is not possible to incorporate critical questions into an argument diagram. This is an important limitation for systems that use argument diagrams, often called argument maps in AI, to model forms of argument such as practical reasoning.
The way to overcome this limitation adopted by the Carneades Argumentation System (CAS) (Gordon, 2010) was to divide the premises into three different kinds: ordinary premises, assumptions and exceptions (Gordon & Walton, 2006, 2011. The ordinary premises are the explicit premises. In the heuristic scheme these would be the goal premise and the means premise. The burden of proof requirement (Prakken, Gordon, & Walton, 2007) for an ordinary premise is on the proponent who put forward the argument. This means that the premise is initially taken to hold, but as soon as an appropriate critical question is asked, the premise no longer holds until the question is answered. Assumptions are treated like the ordinary premises in this regard. Exceptions are treated differently with respect to burden of proof requirements. A critical question that is classified as an exception only defeats the argument if the exception is backed up by additional arguments supporting the claim that the premise does not hold.
Using this approach, more sophisticated versions of the argumentation scheme for practical reasoning can be devised that are more adequate for helping us to evaluate real instances of practical reasoning, arguments of the kind that can be quite complex. Precisely how this has been done will be shown in the rest of the paper using relatively simple examples for purposes of illustration.

Characteristics of an Intelligent Autonomous Agent
It is fundamental to define the concept of an intelligent autonomous agent (IAA), the entity that engages in practical reasoning. The following fourteen characteristics of an intelligent rational agent from (Walton, 2015, pp. 12-13) are summarized in the list below. This list is an extension of the list of eleven characteristics given in (Walton, 1990, pp. 142-143).
1) As an IAA carries out actions and expresses speech acts of certain kinds, such as making assertions, it incurs commitments that can be recorded. It can be inferred from some of these commitments that the agent has formulated or committed itself to a goal.
2) An IAA has the capability for perception and for collecting information from other sources as well, such as reports by witnesses or experts, and can therefore find out about its current circumstances.
3) An IAA has the capability for carrying out actions of a kind that can change its circumstances.
4) An IAA has familiarity with the normal ways things are expected to work in institutions, and has other kinds of common knowledge about the wider world. 5) An IAA can perceive or find out about the consequences of its actions.
6) An IAA can correct its previous or planned actions if it sees that the consequences of those actions are likely to run contrary to its goals. 7) An IAA is capable of grasping how actions to achieve a goal need to fall into an ordered sequence.
8) An IAA can organize a sequence of actions into a hierarchy with different levels of abstraction, from the more general to the more specific and vice versa. 9) An IAA can form hypotheticals about possible future consequences of its actions.
10) An IAA often needs to be flexible in planning by quickly adapting to new information. 11) An IAA will generally keep trying to achieve a goal even if it has previously failed.
12) An IAA has sufficient resources of memory to keep track of its commitments as well as to retain knowledge of the circumstances as they change over time.
13) An IAA has the capability to add new commitments to its previous store of commitments and to retract commitments as needed.
14) An IAA needs to not only be aware of at least some consequences of its past actions, but to keep them in its memory for possible use in future deliberations.
These characteristics of an IAA are fundamentally important for understanding the nature of practical reasoning.  Vol. 8, No. 1;2016 One reason is that they clearly bring out how practical reasoning needs to adapt to changing circumstances of a particular case. In some instances inconsistency of commitments within an IAA's commitment store is considered a defect. The agent may be attacked as inconsistent, or even hypocritical or illogical, as indicated by the existence of the circumstantial ad hominem fallacy identified in logic textbooks. However, inconsistency is not always a defect in intelligent rational deliberation. In a complex action plan, an inconsistency may be overlooked. But once such an inconsistency has been identified, failing to correct it can be an important kind of error. One kind of inconsistency is a logical inconsistency, where one proposition that is part of the plan is logically inconsistent with another proposition that is also part of the plan. Another kind of inconsistency is a practical inconsistency, when the carrying out of one action and plan is incompatible with another action that is also part of the agent's plan, or with an expressed goal of the agent.
In such cases, if an IAA fails to change its plan, or even to scrap the plan entirely, in the face of incoming information showing that the plan is not working, or would lead to disastrous consequences that would go against its goals, that error can be an even more egregious failure to use intelligent practical reasoning. Another feature of an autonomous agent using practical reasoning intelligently (Walton, 1990, p. 143) is the ability to criticize its own actions, as well as others' plans and arguments, and in particular to identify and eliminate any practical inconsistencies in a plan where one action or goal runs contrary to another. The need for this feature in an IAA suggests the need for practical reasoning to not just comprise the actions of a single agent, but to also take into account groups of intelligent agents acting together. The need for an IAA to criticize its own actions, as well as others' plans and arguments, also suggests the usefulness of taking an argumentation approach. But before this technology came on the scene, work on practical reasoning took a different approach.

Different Approaches to Modeling Practical Reasoning
Having looked at an example of practical reasoning, it is necessary now to consider a number of philosophical controversies about the precise form that the argumentation scheme for practical reasoning should take. Probably the first issue that will arise in the reader's mind is whether the word "intention" should be used in the goal premise instead of the word "goal". The widely accepted BDI model uses "intention" (or in other versions "want" or "desire") instead of "goal" in the first premise. The survey of the literature on practical reasoning presented in (Walton, 1990, chapter 1) shows that the BDI model was the dominant model in twentieth-century analytical philosophy. Little has changed in the more recent philosophy literature. Belief, desire and intention are the fundamental constants in nearly all the accounts representing the logical structure of practical reasoning as a type of inference from premises to a conclusion.
The BDI model has also been widely adopted in AI. It is based on the central notion of an agent that reasons towards achieving its intentions based on incoming perceptions that update its beliefs. According to the leading exponents of the BDI model (Bratman, 1987;Bratman, Israel, & Pollack, 1988;Wooldridge, 2002;Paglieri & Castelfranchi, 2005), an agent possesses a set of beliefs that are continually being updated by sensory input coming in from its environment, and a set of desires that are evaluated to form intentions. In the BDI model, the agent's intention is defined as a goal that is persistent over time and not easily given up. Drawing a precise distinction between acceptance and belief has proved to be a hard problem, primarily because there is little basic agreement in analytical philosophy on how to define "belief" (Engel, 2000).
According to the commitment model of argumentation, agents interact with each other verbally in a dialogue structure in which each contributes speech acts (van Eemeren & Grootendorst, 2004). Each party has a commitment set containing the propositions he has accepted, judging by his speech acts in the previous dialogue. As each move is made, commitments are inserted into or retracted from each set according to commitment rules, depending on the type of move each makes. A commitment of the simplest and most basic kind is a proposition that an agent has gone on record as accepting (Hamblin, 1970(Hamblin, , 1971). On the commitment-based approach, practical reasoning is modeled in a dialogue format using an argumentation scheme with a set of critical questions matching the scheme.
The difference between the commitment model and the BDI model is that desires and beliefs are psychological notions internal to an agent, while commitments are statements externally accepted by an agent in a dialogue. The BDI model is more appropriate for psychology, where intentions, beliefs, motivations and other internal mainsprings of action are the central concern. The commitment model has the advantage that it is a more purely logical approach that does not need to directly concern itself with determining an agent's psychological motivations and beliefs. In the remainder of this paper, the commitment approach will be taken, but in most instances it is also possible to utilize the BDI model of practical reasoning if that is the reader's preference.
Next we need to turn to a technical problem in the formulation of the scheme for practical reasoning. We have to www.ccsenet.org/res Vol. 8, No. 1;2016 ask whether the notion of means that is part of the means premise should be more precisely represented as expressing a necessary condition or a sufficient condition of carrying out the action recommended in the conclusion. Audi (1989, p. 86) recognized the distinction between necessary condition schemes and sufficient conditions schemes as important for analyzing practical reasoning. He offered the following example of a sufficient condition scheme (p. 87): "I really need a peaceful visit in the country. Accepting their invitation for a weekend in the Catskills would be a good way to have such a visit, so I'll accept it". According to Audi's interpretation of the example, a weekend in the Catskills would provide a sufficient condition for having a peaceful visit in the country. It may not be a necessary condition however. There might be many other places where one could go to have a peaceful visit in the country. In other instances in the literature on practical reasoning, however, the means refers to a necessary condition. Von Wright (1963) used the following example to illustrate practical reasoning: X wants to reach the train on time; unless X runs he will not reach the train on time; therefore, X must run. In this example, the necessary condition version of the scheme appears to be illustrated. It may well be that the necessary condition version of the scheme is more commonly applicable to examples of practical reasoning in everyday conversational argumentation. But there can also be long chains of practical reasoning combining instances of the necessary condition scheme with the sufficient condition scheme. Because of this problem, the neutral term "means" was used in the heuristic version of the basic scheme given above.

Review of European Studies
Subsequent to the development of these basic approaches to practical reasoning that concentrated on examples of a single agent carrying out its intentions based on its beliefs, there was a literature working on examples of groups of agents acting together to carry out their common goals: see (Tuomela, 2013). The last development was the literature on practical reasoning which was commitment-based and which viewed practical reasoning as a species of argumentation which groups of agents could use to communicate with each other to coordinate their plans and goals: see (Walton, 2015). The usefulness of these last two approaches can be illustrated by the following example.

An Example of Political Deliberation
An example given (Walton, 1998, pp. 169-171) concerned a meeting called to discuss whether or not to bring in no-fault auto insurance in a state, made urgent by the circumstances that insurance rates had increased to such a level that they had become burdensome. This example was based on a longer case study of a town hall meeting described by Lascher (1999). Here we present an analysis of it by using argument diagramming to model the argumentation put forward in the meeting as a connected series of pro and con arguments based on practical reasoning.
How argument mapping can be used to represent some of the arguments in this case study deliberation is shown in the sequence of arguments represented below in the style of CAS. The side advocating the change to a no-fault system used argument from positive consequences when they claimed that changing to a no-fault system would lower the cost of insurance. Assuming that the goal is to reduce the cost of insurance, this argument can be represented in the argument map shown in figure 2, illustrating how practical reasoning is used as a pro argument by one side in a deliberation.

Figure 2. Pro argumentation
This is a linked argument, meaning that both premises go together to support the conclusion. The scheme for practical reasoning has two premises, a goal and a means premise. Both are required to make the argument fit the scheme. If either premise is missing, the argument from practical reasoning would not be valid. Here the term 'valid' is to be interpreted in a sense meaning defeasibly valid (Walton, 1990).
Next it can be shown that the other side uses argumentation to attack one of the premises of the practical reasoning used by the pro side as shown in figure 2. In figure 3, the other side argues that changing to a no-fault system would fail to lower premiums. Figure 3. Con argumentation The argument pattern shown in figure 3 is an example of a premise attack. The minor premise of the practical reasoning scheme is attacked. It is argued that changing to a no-fault system would fail to lower premiums, and since premiums represent the cost of insurance, the conclusion that changing to a no-fault system would lower the cost of insurance is challenged.
But there is a general problem with argument mapping illustrated here. The text boxes in the argument maps only represent propositions, so there appears to be no way that critical question can be represented on an argument map. This problem has been overcome in CAS by representing critical questions as additional premises implicit in an argumentation scheme. The reader can be referred to section 1 above and (Gordon & Walton, 2006) to see how this procedure works. For our purposes here, it is sufficient to observe using this example how a critical question can perform a role similar to that of an undercutter (Pollock, 1995).
One of the critical questions matching the scheme for practical reasoning is the question asking whether there is another means for carrying out the goal. This question CQ 2 is shown at the bottom left in figure 4. The argument from practical reasoning is shown at the top. The critical question, as represented in the argument graph, takes the form of a rebuttal, a counter-argument that attacks the ultimate conclusion, the statement that the action of changing to a no-fault system should be carried out. In other words, what we have at this point is a conflict between a pro argument and a con argument. Unless the one argument can be shown to be stronger than the other, there will be deadlock in the deliberations. The deadlock could be resolved, however, if some additional evidence backing up the one argument or the other could be introduced. In figure 4, the critical question has been represented as a con argument leading by an arrow to the rectangular node representing the conclusion of the pro argument from practical reasoning just above it. As shown in figure 5, in this instance the critical question is backed up by some evidence that supports it. The critical question asks whether there is another way to lower the cost of insurance. The pro argument supporting the critical question is based on the assertion that reform of the insurance industry would succeed in lowering insurance rates. So we can see that in this kind of case an issue concerning burden of proof is addressed. Some critical questions, called exceptions in CAS, only defeat the argument they are addressed to if some evidence is given to back them up. The argument shown in figure 4 is just such a case in point.
The sequence of argumentation is carried a step further by the argument shown in figure 5. In this figure, an additional critical question is asked. It is a critical subquestion concerning alternative means. It is the critical question of whether the other way to reduce the cost of insurance is better than the original one proposed. The first critical question functions as a rebutter attacking the ultimate conclusion, while the second critical question attacks the support for the first one. Here we have an attack on an argument supporting an attack on a claim. It is more of a challenge to model the argumentation in this instance, because we appear to have one critical question attacking another. CQ2 is represented as a question, attacking the conclusion of the prior argument from practical reasoning. But then CQ3 is modeled as a proposition that functions in a premise in an argument that attacks the inferential link between the support given by the proposition at the bottom to the argument supporting the previous critical question. This kind of attack is called an undercutter (Pollock, 1995).
Generally, an argument can be attacked in three ways (Prakken, 2010, p. 169): (1) by attacking one or more premises of the argument, (2) by attacking the inferential link between the premises and conclusion, or (3) by presenting a counter-argument showing that the conclusion of the original argument is not acceptable. The second type of attack is an undercutting of the prior argument. Of course there can also be other ways of attacking an argument, such as making the claim that the argument commits an informal fallacy, for instance the fallacy of begging the question (arguing in a circle). But it is still very useful when constructing argument diagrams to primarily consider these three chief ways of attacking an argument. The argument represented in figure 5 represents two of them, and the previous argument diagram represents the other one.
There can be other ways of using an argument diagram to represent the structure of the practical reasoning in this example, but figure 5 at least illustrates some of the interesting problems that can arise in such cases. In the section below, once a more detailed model of the argumentation scheme for practical reasoning is presented, incorporating the critical questions as different kinds of premises of the scheme, more resources will be available for dealing with these more complex kinds of cases of practical reasoning.

Value-Based Practical Reasoning
The next big step in the evolution of argumentation-based models of practical reasoning was the introduction of the argumentation scheme for value-based practical reasoning by Atkinson and Bench-Capon (2007). They extended the earlier argumentation scheme of Walton (1990Walton ( , 2007 that did not take values into account. The argumentation scheme put forward by Atkinson and Bench-Capon (2007, p. 858) took the following form.
In the current circumstances R We should perform action A Which will result in new circumstances S Which will realize goal G Which will promote some value V.
They offer the following example: "I may diet to lose weight, with the goal of not being overweight, to promote the value of health". On this version of the scheme, the earlier Walton scheme is supplemented to give reasons why the goal and the action are desirable (Atkinson and Bench-Capon, 2007, p. 858). In this example, the value of health gives a reason why the goal of health is desirable and gives a reason why the action of dieting is desirable.
A notable feature of this model is that it expands the list of critical questions to a new list of sixteen (Atkinson Another innovation of this new version of the scheme is that an agent's performing of an action is represented as a movement from one state of affairs to another. The new state of affairs brought into being by practical reasoning of this sort is brought into line with the values accepted by the agent that carried out the action. To allow for extending the analysis to instances where an action is carried out by a group of agents, Atkinson and Bench-Capon (2007, p. 860) introduced an action-based alternating transitions system, made up of a finite set of states, an initial state, a finite set of agents, a finite set of actions, an action precondition function which defines the set of states from which a given action may be executed, a partial transition function that defines the state that would result from the performance of an action, a finite set of atomic propositions, and an interpretation function determining when a given set of propositions is satisfied in each state. Accordingly, the scheme is extended to take into account the agent acting from an initial state by an action that leads to a secondary state representing the outcome of the action.
There is one additional respect in which the value-based version of the scheme is especially interesting and significant for the purposes of the account of practical reasoning described here. It comprises not only a problem formulation and a choice of actions, but also an element of epistemic reasoning because it models the circumstances of the given case as the agent knows them at the time it is ready to carry out its action. These features tie in with the list of requirements for an IAA formulated in section 2 above, and also have implications for applying the practical reasoning scheme to realistic cases of group decision-making by intelligent agents. When applying the scheme, it needs to be presumed that the agent is aware of its circumstances, but also that the agent can be mistaken in this regard. Practical reasoning in real-world applications is defeasible because the agent is not omniscient and its circumstances can change. It may fail to take new circumstances into account that might well lead to a failure to realize its goal. It is for such reasons that the critical questions are important in the argumentation approach. Fairclough and Fairclough (2012) have applied the scheme for practical reasoning, along with argument diagramming techniques, to many cases of cases of political argumentation in current events, including government reports, parliamentary debates, political speeches and online political discussion forums. They used a version of the practical reasoning scheme comparable to the value-based scheme described above. Their work includes a very valuable collection of examples of political discourse in which the scheme has been applied. Applying argumentation schemes to real examples of arguments in natural language discourse, and in particular to political debates, can often be very complex, because it involves not only deliberation, but interest-based negotiation and persuasion dialogue. This complexity wil be explained in section 9.
This work poses some problems of how to study practical reasoning and apply argumentation schemes to the use of it in realistic cases. Here one of these problems is mentioned in passing. Conforming to the Atkinson and Bench-Capon formulation of the scheme for value-based practical reasoning, the Faircloughs' argument diagrams have a text box for the conclusion, another text box for the goal, a third one for the values, and a fourth one for the circumstances of the case. These circumstances are held to be composed of natural facts, as well as social and institutional facts. Institutional facts are said to include the agent's value commitments, for example duties, promises, socially recognized values and norms. They have adopted the approach of Searle (2001) which includes institutional facts under the heading of "facts", and perhaps for this reason in many of their argument diagrams they have included what appear to be values under the heading of circumstances. In one example (Fairclough & Fairclough, 2012, p. 150) the information in the circumstances text box includes distribution of housing benefits by the government described as being out of control and unfair.
However, when applying the argumentation scheme for practical reasoning, values need to be kept separate from circumstances (Atkinson & Bench-Capon, 2007, p. 858). Presumably the IAA has a knowledge base comprising what it knows of the circumstances that it is facing when contemplating a decision to take action to implement its goal. It may be that in some real cases of political argumentation it is very difficult or perhaps even impossible to clearly separate between values and the circumstances that are supposed to be contained as facts in this knowledge base. At any rate this problem is mentioned here as one worthy of attention for further work attempting to apply argumentation schemes to realistic examples.

Related Schemes
We have already seen that one of the critical questions matching the scheme for practical reasoning is concerned with side effects, negative consequences that may come about if an action to achieve a goal is undertaken. In many instances of practical reasoning, a very common form of argument called argument from consequences can be used to attack an instance of practical reasoning. The scheme for this type of argument has two forms. The first one is argument from positive consequences. The use of the term "good" indicates that something is taken to have positive value.
Major Premise: If A is brought about, consequences C will plausibly occur.
Minor Premise: Consequences C are good.
Conclusion: A should be brought about.
Along with this scheme there is a negative variant called argument from negative consequences.

Major Premise: If
A is brought about, consequences C will plausibly occur.
Minor Premise: Consequences C are bad.
Conclusion: A should not be brought about.
Argumentation from consequences offers a reason to accept a proposal for action tentatively, subject to counter-arguments that may be advanced by the other side. For example, it is very common for arguments from negative consequences to be attacked using argument from positive consequences, or vice versa.
One can also see that both variants of argument from consequences are based on the assumption that consequences of an action can be designated as having positive or negative value. However, arguments from positive or negative values can also operate as individual arguments in their own right (Bench-Capon, 2003) independently of argument from consequences. The first argumentation scheme represents the argument from positive value.
Major Premise: If value V is positive, it supports commitment to goal G.
Minor Premise: Value V is positive as judged by agent a.
Conclusion: V is a reason for a to commit to goal G.
The negative counterpart is called argument from negative value.
Major Premise: If value V is negative, it gives a reason to retract commitment to goal G.
Minor Premise: Value V is negative as judged by agent a.
Conclusion: V gives a reason for a to retract commitment to goal G.
Next we need to see how argument from values can be combined with instrumental practical reasoning to yield the following scheme for value-based practical reasoning.
The structure of this kind of argument is displayed in the argument diagram of figure 6. Argument from negative value (-AV) is employed to rebut the practical reasoning argument used to support the conclusion advocating change to a no-fault system. In this case, the con argument at the bottom is used as a rebutter. The con argument (-AV) attacks the conclusion. The plus sign indicates that the practical reasoning is used in this instance as a pro argument. As shown in figure 6, the con argument representing the scheme for argument from negative value acts like a critical question that gives a reason for casting doubt on the argument it opposes.

Instrumental and Value-Based Practical Reasoning
The argumentation scheme for value-based practical reasoning has been formulated within the CAS model as shown below so that it can be used with the CAS argument diagramming tools and the CAS system for evaluating arguments. This version of the scheme, based on the version of Atkinson and Bench-Capon (2007)

premises:
 S1 is the case in the current circumstances.  Performing A in S1 would bring about S2.  G would be realized in S2.  Achieving the goal G would promote the value V.

assumptions:
 V is a legitimate value.  G is a worthy goal.  Action A is possible.

exceptions:
 There exists an action that would bring about S2 more effectively than A.  There exists an action that would realize the goal G more effectively than A.  There exists an action that would promote the value V more effectively than A.  Performing A in S1 would have side-effects which demote V or some other value.
This version clearly displays the CAS method of modeling the critical questions matching a scheme as additional premises called assumptions and exceptions. The assumptions and exceptions can be added in to represent the critical questions as propositions that can be visualized in an argument diagram. This way of modeling a scheme makes it more complex, but also nicely brings out the distinction between the original heuristic form of the scheme we started with, and a more complex form that takes the critical questions into account. Once this more complex form of the scheme is utilized in a system like CAS that enables the user to construct a large argument diagram representing the pro and con argumentation in a given case, a method for evaluating practical reasoning as a species of argumentation is made available.
In some cases values are in the forefront for properly evaluating cases where practical reasoning is used. For example, political debates and deliberations are heavily based on values, including shared values of groups of participants in the debate, and therefore it is necessary to use the value-based version of the scheme for practical reasoning to properly analyze the argumentation in such cases. In other cases values are not in the forefront. For example, consider the case of a man who is trying to fix his printer, because a black line appeared down the middle of every page that was printed (Walton, 2015, p. 149). The man wants to solve this problem so he starts taking the printer apart, and while doing this he downloads instructions from the website of the company who made the printer on how to solve problems with it. He finds that he has to remove the glass plate from the top of the printer, and carry out another sequence of actions so that he can extract and clean another part of the printer that functions along with the glass plate. Here he goes through a procedure that can be called problem-solving, and it is clear that he uses practical reasoning to try to solve this problem. This problem is to fix the printer, that is, to restore it to a state where it no longer prints the black line down the middle of the page. In order to achieve this goal, he carries out a connected sequence of actions of the kind described in the instructions manual. But values are not in the forefront of the practical reasoning in such a case. The practical reasoning can be represented using an argument diagram that applies a simpler scheme than the value-based one above.
The analysis of practical reasoning as applied to examples of this sort suggests a different approach (Walton,  2015). In this approach, a distinction is drawn between instrumental and value-based practical reasoning so that instrumental practical reasoning is taken to be the basic argumentation scheme. Then value-based cases of practical reasoning can be treated by considering values as additional implicit or explicit premises.
Considerations of values can be added on to the basic scheme where required, leading to the use of the value-based scheme.
In line with this approach, the following the instrumental scheme for practical reasoning has been configured to fit the CAS format (Walton, 2015, pp. 93-94).

Argument from Instrumental Practical Reasoning
id: practical-reasoning strict: false direction: pro conclusion: Action A should be performed. premises:  S1 is the case in the current circumstances.  Performing A in S1 would bring about S2.  G would be realized in S2.

assumptions:
 G is possible.  Action A is possible.

exceptions:
 There exists an action that would bring about S2 more effectively than A.  There exists an action that would realize the goal G more effectively than A.  There are intervening actions required to move from the action A to the goal G.  Performing A in S1 would have side-effects that need to be taken into account.  There is another goal G' that is incompatible with G.
These are the two schemes for practical reasoning configured in the CAS format. Once values are taken into account, this additional factor can make the conclusion in a given case more difficult to arrive at. Factual disputes can be difficult to resolve decisively one way or the other by means of pro-con argumentation, but once values are also taken into account, it can make the issue even more difficult to resolve. Further decisions will have to be made about how to order values by priority, and this can sometimes be a difficult task for value-based practical reasoning. In economics, for example, it may be possible to provide a cost-benefit analysis that does not take values into account, but is still useful for purposes of applying practical reasoning.
As the examples treated above show, along with a range of other examples presented in (Walton, 2015), the best way to analyze and evaluate examples of practical reasoning as a form of argumentation is to use the instrumental scheme where values considerations are not in the forefront, but bring in the value-based scheme only in cases where it is important to take values into account. The value-based practical reasoning scheme is more complex than the instrumental scheme. It has additional premises that are not included in the instrumental scheme. Therefore the value-based scheme can be treated as a subspecies of the instrumental scheme, applicable to cases where considerations of value are necessary to take into account in calculating the reasoning used to derive the conclusion that a particular course of action is recommended.
One reason the circumstances are not represented explicitly in the heuristic form of the argumentation scheme for practical reasoning is that they operate as part of the background. An IAA has the capability of being aware of its circumstances, and when changes in its circumstances take place from one state to another, it can react accordingly by modifying its actions. This aspect of taking changing circumstances to account is included in the list of critical questions matching this scheme, but it is not included in the heuristic version of the scheme itself. The two longer versions of the scheme incorporate the critical questions as additional premises that can take the form of assumptions or exceptions, depending on where the burden of proof lies.
In the model of (Walton, 2015), practical reasoning is paradigmatically used in the setting of rational deliberation where a group of agents (or in some instances a single agent who may have some input from other agents) is trying to make a decision on what to do in a given set of circumstances. In this model of rational deliberation, one of the most important properties is for the IAA to be constantly open to changes in its circumstances so that it can modify its practical reasoning accordingly. Therefore the agent's awareness and constant attention to its www.ccsenet.org/res Vol. 8, No. 1;2016 changing circumstances is built into the model of deliberation by the IAA.

Other Resources from AI
There are decision support systems in AI that build on argumentation-theoretic models of deliberation to better integrate with human reasoning. Practical reasoning is also closely related to the technology of planning, a field well developed in AI. Gordon and Karacapilidis (1997) designed a mediation system to assist interested citizens and members of public interest groups to engage in electronic discussions with government officials. Experts, such as city planners, can be included in the discussions. Atkinson, Bench-Capon and McBurney (2004) built the Parmenides Argumentation System that can be used by governments to solicit public input on policies being under consideration. In this system, the policy is formulated in the practical reasoning format, meaning that goals, and the means to achieve them, are put forward and discussed. A succession of screens in the visual interface even presents objections to the proposed policy actions and goals, allowing for argumentation to develop as dialogue between pro and con arguments.
By explaining how AI has moved ahead to develop a wide array of argumentation-based tools and models, it is shown how those of us in the field of argumentation need to look increasingly towards this scientific field both to apply the computational tools from it and to gain important insights into the precise structures of defeasible reasoning and value-based argumentation that we need to work with.
Although early work in argumentation theory concentrated on the critical discussion, a type of persuasion dialogue rightly taken to be central to humanities and speech communication studies on argumentation, there is a need to expand to other types of dialogue, and most notably the deliberation type of dialogue. McBurney, Hitchcock and Parsons (2007) have provided a formal model of deliberation dialogue. Examples of policy decision-making dialogues are modeled as argument graphs in the Carneades Argumentation System (http://carneades.github.com). Its policy analysis menu gives the user a tool for analyzing and comparing the effects of various policy proposals in example cases. As the user is asked questions in order to collect knowledge about the particulars of the case, an argument graph is generated by the system, showing how the policies that are being considered would work out as their consequences are generated.
These examples are especially interesting in that they show the close connection between practical reasoning and argumentation from consequences, and they show very clearly the importance of argumentation from consequences in these kinds of significant deliberations. While it may be true that lowering taxes increases productivity, and may be true that increased productivity is good, it may also be true that lowering taxes decreases equality, and that decreased equality is bad. We all realize that this kind of conflict of opinion has been fundamental to recent elections in the United States in a situation where the division between the Democrats and the Republicans has been sharper as a two-sided conflict than ever before. Argumentation tools for reconstructing and evaluating arguments not only have explanatory power as applied to these situations, but can also take into account the complexity of such discussions, and give us some hope of extracting from them the thread of rational argumentation embedded in them.
AI has provided us with an interesting model of how argument justification can be viewed as a process in which a given argument survives conflicts with counterarguments. It is up to us as argumentation theorists to study how this model fits with our views on how arguments should be evaluated. Before this we have not had such a formal model, and that has been a grave defect in our area if we want to promote it as an interdisciplinary field which has a scientific component.
In deductive logic, we can evaluate an argument in a context-free manner by applying deductive rules of inference to show a given argument to be valid or invalid. This framework for evaluating arguments is monotonic, meaning that an argument always remains valid no matter how many new premises are added. Clearly this framework is inadequate to studying practical reasoning because practical reasoning is inherently open-ended and the closure problem can only be solved by placing it in a procedural context that is open to ne information. As we saw in section 1, practical reasoning presupposes an intelligent autonomous agent that is deliberating and attempting to make a rational decision on what to do in circumstances that are inherently changeable. In order to provide an adequate model of practical reasoning, therefore, we need to move from the deductive logic mindset to the new view of practical rasoning compatible with multiagent systems of the kind now commonly used in AI. This framework takes the form of a dialogue in which agents make decisions by collecting information based on interactions with other agents, for example on the Internet, or in forming a group plan on how to take action based on proposals that can be discussed and evaluated. For this purpose, we need to work with the notion of an intelligent autonomous goal-directed agent that is capable of interacting with other agents in a procedure called a deliberation dialogue.  Walton, Toniolo and Norman (2014) argued that a computational model of deliberation useful for dynamic multiagent systems is best based on a natural concept of deliberation, meaning that it should share certain important features with real examples of natural language deliberation. One important feature of natural deliberation is that it needs to remain open to collecting new information and considering arguments as long as this procedure continues to be fruitful, but at the same time it needs to be closed off once the circumstances require arriving at a timely decision on what to do. Their model was designed to capture the capability of a deliberating agent to update its practical reasoning based on changes of circumstances coming into its knowledge base as the dialogue proceeds. They extended the previous models by adding an open knowledge base. This feature enables the agents to modify their plans using argumentation schemes to improve the flexibility of their plans as new information about the circumstances that is relevant to a plan comes in. This model was designed to stress the need for flexibility in deliberations, so that the plan adopted by a group of agents can quickly be modified to take in new information into account. This feature enhances the adaptability of a plan in relation to circumstances that may be rapidly changing.

Practical Reasoning in Persuasion Dialogue and Deliberation
In typical political deliberations, such as in the case of the town hall meeting described in section 4, the discussion starts out as a deliberation on what to do in a situation requiring choice. In this case the choice was whether to bring in no-fault auto insurance or not. Typically, however, such practical deliberations about what to do shift to epistemic arguments about whether a proposition is true or false. An example is provided by the argument shown in figure 3. On the left side of this argument diagram, practical reasoning was used to support the action to change to a no-fault system. The conclusion is an action, and so this part of the argument is a deliberation about what to do. However the argument on the right of figure 3 supported the premise that changing to a no-fault system would lower the cost of insurance by using a con argument with a premise stating that changing to a no-fault system would fail to lower premiums. In this argument, the conclusion is a proposition which could be true or false. So this part of the argument is epistemic. It is about the alleged circumstances of the case. Here the arguer is trying to persuade the audience not to accept the proposition that changing to a no-fault system lower the cost of insurance by making the claim that changing to a no-fault system would fail to lower premiums. This proposition in turn could be supported or attacked by further factual claims. Here we have a shift in the same argument from one type of dialogue to another.
In some cases it is not straightforward to determine whether a given argument should be best evaluated as part of the one type of dialogue or the other. Practical reasoning is most typically used in deliberation. But there can be persuasion over action (a species of persuasion dialogue). However, there is a pervasive problem of building a firm and precise basis for distinguishing between deliberation and persuasion over action. Consider the following debate topics from Debatepedia.
• Should there be a ban on sales of violent video games to minors?
• Should there be mandatory ultrasounds before abortions?
• Should governments legalise all drugs?
• Should the Roman Catholic Church change its current position of forbidding the use of contraception?
In each case, the topic of the debate concerns a decision to take action, suggesting deliberation. But the decisions are not being made by the debaters themselves. These dialogues were classified as instances of persuasion over action by (Atkinson et al., 2013). But it may seem problematic to make such a judgment until there is more clarity about the difference between persuasion dialogue and deliberation. To sort this out, let us begin with the critical discussion type of dialogue, a type of dialogue that has already been clearly identified and defined (van Eemeren & Grootendorst, 2004).
The goal of a critical discussion, a type of persuasion dialogue, is to resolve a conflict of opinions by examining the pro and con arguments on both sides of the conflict (van Eemeren & Grootendorst, 2004). Hence there is a burden of persuasion in this type of dialogue (often called a burden of proof). There are basically three stages in a persuasion dialogue. First there is an opening stage where the participants agree to take part, the understanding being that they will take turns putting forward arguments, to see which side can put forward the stronger argument. The opening stage also includes what is called a confrontation stage, where the conflict of opinions that has provoked the argument is stated. One side, the proponent, has made a particular claim, called its thesis to be proved, while the other side, the respondent, either advocates an opposed thesis, or at least has expressed doubts about the tenability of the first party's thesis. At the opening stage, what is called the burden of persuasion www.ccsenet.org/res Vol. 8, No. 1;2016 is set in place, including the standard of proof to be met, and which side has this burden of persuasion. Next, in the argumentation stage, both parties have a chance to put forward their strongest arguments and criticize the arguments of the other party. Finally, there is the closing stage where it is determined which side has won the argument, based on fulfillment of the burden of persuasion set at the opening stage.

Review of European Studies
The sequence of argumentation through the three stages in a persuasion dialogue is graphically illustrated in figure 7.

Figure 7. Sequence of argumentation in a persuasion dialogue
As shown in figure 7, the conflict is resolved at the closing stage once it has been shown by the end of the argumentation stage that one side or the other has met its burden of persuasion. Persuasion dialogue is by its nature highly adversarial. Each side uses strategic maneuvering to prove its own claims and to systematically attack the arguments of the opposed side.
Deliberation is sometimes viewed as a procedure by which a single agent arrives at an intelligent conclusion on how to act. For example a man trying to repair his car when it won't start would not normally be seen as engaging in a deliberation dialogue with other agents. But he might ask other people for advice, and he might even reach a decision about what the best explanation of the failure is, as the basis of moving forward, by examining the pros and cons of several hypotheses. Also, the examples of the study of practical reasoning in AI systems such as planning show that group deliberations are also common. For example, it is common to take the advice of an expert in a political deliberation, to have family deliberations, to have deliberations on how to carry out a task, and to have policy deliberations.
In deliberation, the opening stage is the formulation of an issue about what action to take to achieve some goal, not a position or claim to be proved (McBurney, Hitchcock, & Parsons, 2007). The goal of a deliberation dialogue is to make a decision on what to do in a situation where the circumstances require action of some sort, even where doing nothing is a course of action with consequences. In the opening stage a problem is posed concerning which course of action to take in a situation demanding choice. Following the opening stage there is a brainstorming stage where the participants try to come up with proposals on how to solve the problem. Once these proposals have been formulated, a decision has to be made on which is the best one. At this stage I am not just trying to refute my co-deliberator's proposal, or show that my own proposal is the best one. I may decide that my co-deliberator's proposal is better, based on the arguments that have been made, and advocate that proposal. This does not mean that I have lost the deliberation dialogue, as it would if the discussion were a persuasion dialogue. There is no penalty for failure to prove one's claim, i.e. no burden of persuasion.
In parallel to the sequence of argumentation in a persuasion dialogue displayed in figure 10, the sequence of argumentation making up the structure of the deliberation dialogue is illustrated in figure 8. The opening, argumentation and closing stages for the two types of dialogue can be compared and contrasted by viewing figures 7 and 8.

Figure 8. Sequence of argumentation in a deliberation dialogue
The sequence of argumentation shown in figure 8 is simplified in order to provide a contrast with the sequence of argumentation in persuasion dialogue given in figure 7. However, it can be added that part of the argumentation stage that needs to be inserted is the brainstorming phase where the participants in the deliberation formulate proposals before they put them forward and evaluate them (Walton, 2015). Now we can see how a deliberation dialogue is different in its basic structure from a persuasion dialogue. The problem of a deliberation dialogue is to find the best course of action from among the available alternatives. In a persuasion dialogue the goal of each side is to satisfy its burden of proof. However there is no burden of proof in deliberation dialogue. It is not about proving or disproving anything (Walton, Toniolo, & Norman, 2014). It is about finding a proposal that is the best thing to do in the circumstances. In a persuasion dialogue the main burden of proof set at the opening stage is the burden of persuasion. It sets what proposition is to be proved true or false by each party. A party is expected to prove the main claim that he or she makes, and failure to do so means loss of the dialogue to the other party.
As suggested by the MedX example, practical reasoning is used in persuasion attempts (ads) where it is used to try to persuade an audience to buy a product. In this case the pharmaceutical company advertising the product is trying to persuade the audience, that is the readership of the ad, to buy the product. But the audience, the people reading the ad who are deciding whether it is a good idea to buy the product or not, are engaging in deliberation. Thus in many cases persuasion dialogue is combined with deliberation dialogue. In some cases when studying argumentation it does not matter whether the context of use of the argument is a persuasion dialogue or a deliberation dialogue. But in other cases it does matter, as indicated by the presence of a burden of persuasion in the one type of dialogue but not in the other (Walton, Toniolo, & Norman, 2014).
We see through the examples of argumentation reconstructed by Gordon, Prakken and Walton (2007) that even though they are examples involving public discussions on what to do that have legal ramifications, they are clearly examples of deliberation dialogue. Legal court proceedings are generally instances of persuasion dialogue, where the plaintiff or prosecutor has the burden of proof and must persuade the judge or jury, who makes a decision on whether the burden of proof has been met. In deliberation dialogue, in contrast, the starting point of the dialogue is an issue about what action to take to achieve some goal. It is not about resolving a conflict of opinions or attacking or supporting the claim that some proposition is true. The party who raises the issue does not have a burden of persuasion. Indeed, once proposals have been put forward, a party may actually prefer some proposal other than his or her own once the arguments about the pros and cons have been exchanged (Atkinson et al., 2013).
by Gordon and Karacapilidis (1997) are instances of deliberations rather than instances of persuasion dialogue. They are based on an opening stage in which there is incomplete and inconsistent information in the knowledge base, and where the space of possible solutions is not known in advance. These are called wicked problems in computer science. They are characterized by lack of resources, including time and money, to solve the problem by collecting a mass of relevant data. However, practical reasoning and argumentation from consequences can be brought to bear in such situations by representing the mass of evidence in a given case using argument mapping tools. In an argumentation model such as CAS, the arguments pro and con on both sides are weighed against each other, and a conclusion is derived on a balance of considerations using standards of proof. Computational tools, including CAS, have both developed and utilized practical reasoning, as well as argumentation from positive and negative values, and argumentation from consequences.
During the part of the deliberation dialogue in which proposals are being evaluated, factual evidence needs to be brought to bear. In this part of the sequence of argumentation, therefore, persuasion dialogue, information seeking dialogue, and other types of dialogue need to be embedded into the deliberation dialogue. Such an embedding means that there will be a temporary shift from one type of dialogue to another (Walton, Toniolo, & Norman, 2014;Walton, 2015). There is no space in this paper to discuss dialectical shifts of this sort, but a common sort of example can be used to illustrate in general how they take place and why they are necessary.
Let's say that in a debate in a legislative assembly the decision to be made is whether to pass a bill to install a new dam. Arguments are put forward by both sides. One side argues that such a dam will cost too much, and will have bad ecological consequences. The other side argues that the dam is badly needed to produce energy. A lot of facts about the specifics of the dam and the area around it are needed to reasonably evaluate these opposed arguments. The assembly calls in experts in hydraulics engineering, ecology, economics and agriculture, to testify on these matters. Here the dialogue needs to shift from deliberation to a phase of expert consultation where the experts are questioned and information gained from hearing and questioning them is used as a body of evidence needed to evaluate the proposals made in the deliberation dialogue. The experts provide information about the circumstances.

Conclusions
This paper has explained the distinction between instrumental practical reasoning and value-based practical reasoning and has shown how practical reasoning has its own distinctive argumentation schemes. It showed that there is a cluster of argumentation schemes surrounding the basic instrumental scheme for practical reasoning, and proposed that one direction of research is that of classifying the schemes to get a better idea of how to distinguish between them. It was shown how this family of schemes should be configured, and how each has an appropriate set of critical questions matching it. It was shown that it is this kind of group deliberation setting that has been the most frequent utilization of practical reasoning. It was shown how a distinction needs to be drawn between the deliberation type of dialogue, where the goal is to arrive at a decision to take action, and the persuasion type of dialogue where the initiating situation is a conflict of opinions, and each side tries to persuade the other to come to accept its point of view. It was shown how the use of argument mapping is an extremely useful tool to reveal the structure of the argumentation in such cases. It was shown how such an evaluation can be carried out using an argument diagram structure in which the components of the practical reasoning in the case are represented as premises, conclusions, and inferential links that fit argumentation schemes.
The philosophical differences between the various versions of the argumentation scheme for practical reasoning that have been put forward were explained and discussed. In particular, it was shown how the BDI model explains the internal psychology of how an agent uses practical reasoning to arrive at a decision on how to act, based on its beliefs desires and intentions. In contrast it was shown how the commitment model embeds practical reasoning in a formal dialogue model in which two agents take part in an orderly conversational exchange in which they reason with each other. In this model, a goal is not modeled as an intention, but as the commitment that an agent has made by forming a plan or by verbally declaring some proposition as a stated goal. The literature on practical reasoning is split between these two models, but the BDI model has long been the dominant one in the past, and still continues to dominate in the philosophical literature. The commitment model, however, has been adopted and applied in recent work in the area of computational dialectics in AI, and has also been advocated in the field of argumentation studies by van Eemeren and Grootendorst (2004).
Now we can bring everything together in the argumentation model to show how practical reasoning works by a chaining together of actions and goals towards an ultimate goal in a goal-directed deliberation dialogue with an opening and a closing stage.  Figure 9. Operation chart for practical reasoning To appreciate how everything can now be brought together, we have to look back at the fourteen characteristics of an intelligent autonomous agent listed in section 2. We also need to recall that on the argumentation model, practical reasoning is viewed as a dialogue sequence with an opening and a closing stage. Figure 9 represents the sequence of argumentation in a simplified manner so that the reader can grasp how the procedure begins from the start point at the opening stage and leads to an endpoint when the sequence of deliberations is closed off.
In figure 9, the start point at the rightmost part of the figure represents the initial situation at the opening stage where an agent or group of agents is confronted with the problem of deciding what to do in a given set of circumstances. The ultimate goal is G1, represented at the left side of figure 9. If goal G1 is achieved, the dialogue stops. However, there may be many other practical reasons why the dialogue stops. There may be a conflict of goals. There may be side effects of some action that is required to achieve the goal that are so negative that the agent must give up, at least for the time being, in persisting towards achieving the goal.
The entire sequence of chained goals and actions may be very complicated in some cases, involving a lengthy series of actions, goals and changes of goals and actions as the agent receives new information. Through a process of feedback the agent must seek different alternative actions and consider goals other than the main goal (G1 in figure 8) that turn out to be involved in the sequence (Walton, 2015). But let's say that the agent starts out with action A1, which it needs to carry out in order to reach the possibility of carrying out action A2. Then let's say that the agent needs to take the next step of carrying out action A3 if it is to achieve the goal G1. But then the agent receives new information reporting negative consequences of A3. Once it becomes aware of these negative consequences of A3 it sees there is a conflict with one of its other goals G2. But then let's say that the agent does some further calculations and comes to the conclusion that goal G1 should be given priority over goal G2. On this basis the agent can move ahead through a further sequence of actions toward the ultimate action A n that will result in the carrying out of the goal G1. Once the goal G1 is achieved, the procedure comes to a stop. However, this end of the deliberation procedure is only tentative, assuming that it is possible for still further information to come in to the agent that might give it reasons against carrying out G1.