Value Perception of Information Sources in the Context of Learning

Abstract Information sources require consumers to use them in order to evaluate their quality, meaning that they are experience goods. The value perceived before acquisition and use may be different from the value obtained by actual use. Understanding the value perception gap is likely to inform more efficient selection of information sources. The current research studies the value gap in a learning situation. We examine information value perceptions before and after experiencing information in an experiment with 113 software engineers engaged in a problem-based learning task while using and evaluating three types of information sources: supportive, reflective and reciprocal. The results indicate that before using an information source, the subjective value for supportive information is lower than for reflective information. In addition, 55% of the participants preferred to obtain information when presented with a choice. After using an information source no correlation was observed between perceived value of information before and after the use of information source (value gap); participants assigned a higher user experience (UX) value to reflective and reciprocal information than to supportive information; positive correlation between UX value and revealed information value; positive correlation between learning achievement and revealed information value; Reciprocal information is associated with higher learning achievement than reflective and supportive; use of information led to higher learning achievement than avoidance of information. Reciprocal information supports high achievement in software engineering informal learning. Reflective information is valued higher than supportive information sources. If supportive information is essential, learning environments designers should invest heavily in interface design combining reciprocal and reflective elements, such as forums and "try it yourself", respectively


Introduction
Information is known as an 'experience good' meaning that it must be used before actual value unfolds. Herein lays an inspection paradox -a user selects an information source without knowing the actual value of information to oneself, a value revealed upon use. The user cannot return information after consuming it claiming to revert to an uninformed state. This raises a need to study the gap between value perceptions before and after using information. The rationale is to observe the nature of the gap and make recommendations on how to narrow it in order to improve the information usage experience. We study this gap in the context of learning. Information sources are needed for learning. Specifically, we study learning in the area of software engineering because this area combines a wealth of available information sources and a tradition of informal learning. Learners engage in problem-solving while referring to information sources. The massive use of online information as assistance to solve problem-based tasks is prevalent in software engineering education (Moreno, Reisslein & Ozogul, 2009); yet, while the context in this research is software engineering, we believe that insight may be applied to other learning fields. The purpose of this study is to understand the relationship between value perceptions before and during learning and tie those perceptions to source type and learning achievement. This understanding may aid in recommending information sources and interface design to planners of learning environments.
The theoretical background that follows ties together theories of information value perception and software education in the context of adult informal learning. We explain the notion of experience-based value perception and make a novel connection to the concept of user-experience in distributed learning environments.
The study is based on an experiment with 113 software engineers performing a problem-based learning assignment. Three types of information sources served the learners in their assignments: supportive, reflective, and reciprocal. Three value perceptions were assessed: subjective value before using the information sources and both revealed and user-experience values after the use of information.
The following sections will unpack the concepts mentioned in this research and describe the interaction between learners and information: how does information value perception change during learning and how does information value perception relate to information source types and learning achievements?

Literature Review
The section provides a background about adult informal learning that relates to information science in order to explore a complete cycle of events during adult informal learning. Specifically, we examine aspects of information value perceptions before, during and after experiencing information for a problem-based learning (PBL) assignment. PBL is an instructional learner-centered approach, which empowers the learner to conduct research, integrate practice, theory and knowledge to develop a solution to a defined problem (Savery, 2015). We allude to types of information sources as well as to the interactive learning environment where the learning takes place.

Information and user-experience value perceptions
The value of information has been studied from economic and psychological perspectives (Repo, 1986;Raban 2007;Linde & Stock, 2011). Economists consider market value as the price consumers are willing to pay to receive a service or good. They define the value of information based on market value, supply and demand forces, resource allocation, costs and expenses, and productivity. On the other hand, psychologists contribute to the definition of the value of information by considering factors such as emotional, social and ethical norms. Willingness-to-pay (WTP) is a fairly common measure of subjective value which combines the economic and psychological approaches to allocating value to information. WTP is a numeric representation of subjective value, which complements other forms of value assessment explained in the next sections. Repo (1986) analyzed the various psychological and economic approaches and defined the term "valuein-use", which represents how helpful was information in making a decision. Value-in-use distinguishes between the users' perceived value before and after the use of information. The gap between what the user expects and the outcome's perceived value is the result of the actual experience with the information. An experience is subjective; therefore, the experience-value is subjective.
Products or services, including information, which require consumers to use them in order to evaluate their quality, are experience goods (Nelson, 1970;Shapiro & Varian, 1999;Van Alstyne, 1999). For example, information sources such as textual tutorials can only be evaluated after reading and experiencing them. This study is a first attempt at quantifying the experiential value of information and examining its relation to pre-use value perception as well as post use outcomes.
For the present research, we use three terms: subjective value, which is the WTP observed before the use of information; and revealed and user-experience values, which are observed after using the information Brought to you by | University of Haifa Authenticated Download Date | 8/2/18 7:50 AM referring to the total value perception on a 5-star rating scale, and aspects of interface quality assessed by questionnaire, respectively.
In the context of education, learning is a form of experience where information is used specifically to acquire knowledge, skill, or understanding. The learning experience viz-a-viz information value is described in the next sections in three stages: (a) Learners' subjective value perception prior to the learning experience; (b) The experience itself, including learner-information interaction in a distributed learning environment; and (c) Revealed value perceived after the experience.

Prior to learning experience -subjective value perception
In information-enhanced learning environments learners are consumers who select and acquire their desired information based on their subjective preferences and value perception. This corresponds well with the learner-centered approach.
An important factor, which may influence the decision to purchase an information source, is its price, which can be zero (free), fixed or negotiable. Lin, Hsu & Chen. (2013) suggested that "free mentality", which is the consumer's expectation to receive information goods for free on the Internet, has been ironed into some consumers' minds. This mentality reduces consumers' willingness-to-pay (WTP) for online information. However, according to Lin et al. (2013), perceived benefits provided by online information sources reduce the impact of the "free mentality". In our experiment, we provide learners with a playmoney budget in order to elicit their subjective value (WTP) for the various sources of information offered to them for the problem-based assignments. WTP provides a behavioral assessment of value perception (Horowitz & McConnell, 2002) and as such it is an indicator of preferences. It is not seen as an indication of actual price but as a value representation to be used comparatively.
The next section discusses the experience of using types of information during learning.

Learning Experience and Learner-Interface Interaction
During the learning experience, the learner interacts with sources of information and with the environment delivering the information via its interface. The interface is part of the user's experience. The environment in which the learners interact with the information has an essential impact on the way this information is evaluated (Ariely, 2000). User experience (UX) in distributed learning environments considers the subjective, complex and dynamic state of the user. UX is the result of three integrated factors: (1) the user's initial state; (2) the interface, including its design, usability, and functionality; and (3) the context in which the interaction occurs (Hassenzahl & Tractinsky, 2006). Wu et al. (2011), suggest that aesthetics significantly influence user experience and that layout aesthetics and textual information affect visual quality evaluation. According to Iiyoshi and Kumar (2008), UX is, unfortunately, one of the last considerations of many educational technology projects. Yet, technology's look-and-feel is important for community adoption. According to Norman (1988), attractive things simply work better. Tools that engage and are enjoyable to use, enhance creative thinking and problem solving (Barzilai & Blau, 2014).
Adaptive information sources enhance learning experience in e-learning systems (Klašnja-Milićević, Vesin & Ivanović, 2018). However, before learning, a learner must decide to select and use information sources, a decision which reflects the subjective value of information. Measuring and understanding learners' information value perception will enable adaptation of information sources to users' preferences and thereby enhance learning. Moreover, in the context of enhancing consumption of information, recent research suggests that consumers' engagement relates to their experience with information sources (Grinberg, 2018). Design for experience recognizes that all participants rapidly form opinions regarding what information sources are interesting, helpful and worth their attention.
Brought to you by | University of Haifa Authenticated Download Date | 8/2/18 7:50 AM Learners' user-experience value is measured after using the information for problem solving, as is the revealed information value. The following section describes the measurements of both values.

Post experience -evaluating revealed and experience values
Revealed value reflects the value accumulated during the experience of using a source of information for learning. The revealed value is a mental account of the subjective value and the learning experience. Revealed value is measured on a five-star rating scale in this research.
User experience (UX) value comprises four types of attributes which are contained in the AttrakDiff 2 questionnaire (Hassenzahl, Burmester & Koller, 2003): (1) Pragmatic attributes measure how easy it is for a user to finalize the task in the tested environment; (2) Hedonic Stimulation measures the level of the user's personal growth encouragement, including skills and knowledge development; (3) Hedonic identification attributes assess how users identify with the product. People communicate their identity through things they use and own. In the context of software engineering education, GitHub (a collaborative code sharing repository) and StackOverflow (a questions and answers forum for programmers) are examples of services that communicate a strong identity (Iqbal, 2013); (4) Attraction measures the global appeal of the interface. Livingstone (2001) defined self-directed informal learning as "all activities involving the pursuit of understanding, knowledge or skill which occurs without the presence of externally imposed curricular criteria". According to Livingstone, learners select their desired information assistance and sources for pursuit of knowledge. They evaluate the learning activity's achievements corresponding to their subjective experiences. They determine their informal learning objectives, means and process of acquisition. For example, in the domain of software engineering, informal learning is prevalent and there is an abundance of information sources of various kinds, from long textual specifications, through interactive forums to short demos following code examples. Engineers learning via distributed learning environments can choose whether to read detailed specifications, interact with other learners in professional forums, or explore relevant code samples. The common factor to these examples is that software engineering learners control their information source choices in the pursuit of knowledge.

Informal software engineering education
According to Norman and Spohrer's (1996) learner-centered approach, learners learn best when motivated to seek out new knowledge and skills in order to solve given problems. Therefore, focusing software engineering education around a set of realistic motivating problems will result in exploration encouragement and learning. In order to solve the problems, learners choose the information sources according to their subjective preferences and value perception. We measure value perceptions before and after PBL in order to better understand how learners select information sources. In the present study, learning is assessed by measuring achievement in a brief knowledge test and assigning each student a grade. By comparing the learning grades of students who used information sources to those who performed the task without access to information we are able to show how information contributes to learning.

Classifying information sources used during distributed learning
When experiencing the information sources, learners strive to achieve their learning goals. The experiential learning process enables participants to evaluate the learning activity and the information used, as shown in Figure 1  The core assumption of the current research is that different types of information are associated with varying subjective, user-experience and revealed values. Information types may be categorized in order to plan and design effective learning via distributed learning environments. The concept of scaffolding aids in categorizing information sources.
Scaffolding refers to the need for support in the learning process. McLoughlin (2002) claimed that scaffolding is a method for developing new skills and understanding because they are task/problem solving oriented and because they give the learner more control of the learning process. Mbogo, Blake and Suleman (2013) analyzed challenges reported by engineers and offered supportive and reflective scaffolds to overcome challenges such as, confusion and misunderstanding of programming rules, constructing logic from existing programs, and unclear error messages when debugging programs. In the learning process, information sources are a form of scaffolding along with help seeking (Puustinen & Rouet, 2009). Information sources support learner independence and self-paced learning, but they are less adaptable to learner needs than teachers or human experts. However, information sources come in various forms offering varying degrees of potential adaptability to learner needs. We apply definitions from scaffoldbased learning theories to categorize information types.
Jackson, Krajcik and Soloway (1998) describe supportive and reflective scaffolding. Supportive scaffolding is assistance for doing the task, offering advice and encouragement, such as modeling, providing structure and guidelines. In the field of software engineering, an example for supportive scaffolding is online standards specification documents which describe rules, syntax and definitions for programing languages. Reflective scaffolding supports thinking about the task, including planning and evaluating. Reflective scaffolding includes complex code examples coupled with demos or tutorials. The learner can observe the demo, understand the steps and logic of the code and learn from the textual tutorial. Holton and Clarke (2006) describe the reciprocal scaffolding, where at least two people cooperate in performing a task. This mechanism was also defined by Chapanis (1988) as interactive communication referring to a process involving at least two participants, where the content of a message includes prior messages viewed or posted by other participants. Reciprocal scaffolding occurs in online forums for programmers, where each programmer may contribute to an ongoing discussion. Table 1 summarizes how information types are applied to various task of PBL in software engineering.

Research Questions and Hypothesis
The purpose of the study is to examine the effect of subjective, revealed and UX values assigned to three types of information sources (supportive, reflective, reciprocal) during a software engineering learning experience on the learning experience and outcome. The research building blocks are expressed in the hypotheses below and are shown in Figure 2.
Based on the experiential nature of value perceptions, there is no reason to expect differences in values assigned a priori per type of information, therefore: H1a: Reflective, reciprocal and supportive information types have equivalent subjective values of information.
Every learner will experience the usage of information differently. Some learners may perceive to gain much value from reading information while others will feel that the information adds little to their ability to cope with the task, therefore: H1b: Subjective value of information type is not related to the revealed value.
Using reflective and reciprocal information types, consumers interact with the interface, and actively try it. The overall evaluation of the user experience using these information types is expected to be more favorable than for the formal text of supportive information, therefore: H2a: Reflective and reciprocal information types are associated with higher UX value than the supportive information type.
Since difference in value before and after the experience is the result of the experience itself, better user experience is expected to reveal higher information value, and vice versa, therefore: H2b: Higher UX value is associated with higher revealed value for all information types Information sources can assist learners' learning process. We assume that learners evaluate the value-inuse for the learning process. The information sources contribution is evaluated, therefore: H3a: Higher revealed value is associated with higher learning achievement.
In addition, based on the above, we hypothesize that: H3b: Reflective and reciprocal information types are associated with higher learning achievement than supportive information.

H4:
Higher learning achievement is associated with participants who use information sources as part of their learning process compared with non-users of information.
H4 does not appear in Figure 2 because it refers to participants who did not use information for their learning process.
Brought to you by | University of Haifa Authenticated Download Date | 8/2/18 7:50 AM

Method
The purpose of this section is to present the research design, data collection method, research population. To that end, the research design is laboratory experiment, where participants are software engineers. The domain expertise was held constant and information type was controlled. The subjective and revealed values of information were measured, as is the UX value of the entire experience. Usability testing of the interface holding the information and the information itself was also carried out.
Subjects: Participants were undergraduate students in their final year in software engineering, specializing in Web engineering, with two years of experience in Web development techniques; or junior Web developers with one-year experience as Web developers. Therefore, all participants had equivalent preliminary Web technology knowledge in the area of Web programming. 113 participants took part in the experiment. Of those, 77 participants were students and the rest were junior Web developers. Independentsamples t-tests were conducted to compare students and junior Web developers in all dependent variables. No significant difference was found and so the data were merged for hypothesis testing (t=.088, p=.671).
Variables: The independent variable in this research is information type. Three types were offered to the participants: supportive, reciprocal and reflective; one type per question, as described in Table 1.
The dependent variables were: -Subjective value prior to use of information -measured by willingness-to-pay (WTP, from a play money budget). -Revealed value post the use of information -measured by a 5-star rating scale.
-UX value -measured by the AttrakDiff scale described below.
-Grade -learning achievement score explained below and in Table 3.
The AttrakDiff questionnaire includes 28 rating questions for each information type as well as for the interface, meaning that learners answered 28 by 3 rating questions on a bipolar scale with opposing adjectives at each end.
Procedure: an online platform (Survey Gizmo at http://www.surveygizmo.com/) was customized to run the PBL assignment and the information sources value assessment scheme. In one session, participants were asked to answer five challenging questions in the domain of software engineering, specifically in Web applications development. Due to their complexity and the level of their depth, the questions required informal learning (this was verified in a pilot experiment with 21 additional participants). The ability to overcome the challenge and find solutions to the given practical problems requires learning which is not part of the standard knowledge of Web development. Pre-coordination between problem and information source places a constraint on random assignment needed in experimental setups. However, the advanced nature of the problems demanded special attention to offering reliable information per problem (the problems are displayed in Appendix A). Inherent preferences were elicited in the question that offered 3 information sources. In addition, we implemented two control conditions: One question offered no information sources at all and one question offered all three types for the learners to choose, as described in Table 2 below. Table 2 outlines the allocation of information types to questions. Participants received identical questions, but presented in random order. Participants received enough play money which was used to submit bids and purchase information sources according to their WTP. Bids were compared against a random market price unknown to the participants as a proxy for the Becker-Degroot-Marschack incentive-compatible bidding procedure (Becker, DeGroot, & Marschak, 1964). For each information source, the participants could submit up to three bids to avoid excessive guessing. All bids were logged into a database. Only bids that equaled or exceeded the market price granted access to information. Payment was deducted from the digital purse. Otherwise, the participants completed the question without the designated information. Once experiencing the information source, participants were asked to answer the assignment problems and rate the revealed value of information they experienced. For participants who used information, the experiment concluded by a post-experiment online questionnaire evaluating the participants' UX value (AttrakDiff 2). For each information source they used, participants were asked to evaluate its user-experience via the online questionnaire. Predefined online questionnaires were published for each information source: w3c, w3schools and Stackoverflow. Assignment grading scheme: each multiple-choice question had two wrong answers, and two levels of correct answers, based on a professional solution. Right answers won 10 or 60 points, according to the solution's professional level. The most recommended solution received the highest score of 60 points. A wrong answer resulted in deducting -100 points from the score. Grading was explained to the participants prior to the experimental activity.
Revealed value was assessed after using the information when participants rated the information sources on a scale of 1-5 stars. Score and cash balance were displayed to learners after each answer, before submitting their rating for the information source used.
Each participant was able to decide whether to purchase the information or try to answer based on their personal knowledge. They also had the option to search Google independently. Table 3 demonstrates a sample flow where the learner did not purchase any information in the first question. The answer was wrong and 100 points were deducted from the user's score. On the second question, the learner paid forty virtual coins and rated the information. The answer earned the learner 10 points. The rating is not part of the calculation.   Figure 3 provides a schematic summary of the experimental setup and the variables.

Results
Tables 4 and 5 provide descriptive statistics of the dependent variables for each type of information (independent variable).

Control Questions
Two questions were added as controls: One offered no information sources at all, and the second offered all three types of information sources for learners to choose according to their preferences. Question order was random.
The results in Table 4 show that the mean grade in the question without any information sources was 7.88 (n=113) while the mean grade in the question with all three types was 29.35 (n=62), although only 55% of the participants selected an information source in order to answer this question.
The revealed value post experiencing the information sources types in the question with all three types shows that the supportive information was rated an average of 2.32 (n=28), while the reflective was rated an average of 3.48 (n=37); and the reciprocal type was rated an average of 3.05 (n=39).

Relation between information type and subjective information value
H1a results: A non-parametric statistical test was performed because the data is asymmetric, not normally distributed. A Kruskal-Wallis test was conducted to evaluate differences among the three information types (Supportive, Reflective, and Reciprocal) on median change in information subjective value (the price participants were willing to pay to get access to the information). The test was statistically significant χ 2 (2, 113) = 12.72, p = .0017.
Follow-up post-hoc tests were conducted to evaluate pairwise differences among the three information types, controlling the False Discovery Rate to correct for multiple comparisons. The results of these tests indicated a statistically significant difference between the supportive information type and the reflective information type (mean subjective value: 44.84 and 71.14 respectively, as per Table 4).
Brought to you by | University of Haifa Authenticated Download Date | 8/2/18 7:50 AM Hypothesis H1a was partially accepted: While reflective and reciprocal information types have equivalent subjective values; the supportive type has a statistically significant lower value than reflective information.

Correlation between subjective and revealed information value, by information type
H1b results: Spearman's Correlation between the subjective and revealed values by information type showed that no statistically significant effect. In other words, there is no statistically significant correlation between subjective and revealed values of information, per information type.

Relations between information type and UX value
H2a results: A Kruskal-Wallis test was conducted to evaluate differences among the three information types (Supportive, Reflective, and Reciprocal) on median change in UX value. The test was statistically significant χ 2 (2, 94) =12.14, p=.002. Post hoc tests were conducted to evaluate pairwise differences among the three information types, controlling the False Discovery Rate to correct for multiple comparisons. The results of these tests indicated a statistically significant difference between the supportive information type and both the reciprocal and reflective information types (mean UX values: 91.46,114.53 and 114.76 respectively). UX units of measurements are points, which are the sum of word-pairs ranking values from AttrakDiff 2 questionnaire.
Hypothesis H2a was accepted: Participants valued their UX using the reflective and reciprocal information types better than the experience using the supportive type.

Correlations between UX value and revealed information value
H2b results: There is a positive correlation between UX value and revealed information value. Spearman's correlation test was conducted between UX value and revealed information value (revealed value was the rating participants assigned to each information type on a 1-5-star scale). Results show positive correlation, N=157, r=0.24, p = .0027.
Hypothesis H2b is accepted: weak positive correlation between UX and revealed value was found.

Correlations between revealed information value and grade
H3a results: Spearman's correlation test was conducted between revealed information value and grade (objective learning achievement). Results show positive correlation, N=165, r=0.46, p<.0001. Hypothesis H3a is accepted: there is a positive correlation between revealed information value and learning achievement.

Relations between information type and grade
H3b results: Information type is a categorical variable and the grade variable has only three levels, therefore, a chi-square test of independence was performed to examine the relation between information type and grade (objective learning achievement). The relation between these variables was statistically significant, X 2 (2, 170) = 36.53, p<.0001. Post hoc tests were conducted to evaluate pairwise differences among the three information types, controlling the False Discovery Rate to correct for multiple comparisons. The results of these tests indicated a statistically significant difference between both the supportive and reflective information types compared to the reciprocal information type.
Hypothesis H3b was partially accepted: Reciprocal information type is associated with higher learning achievement than reflective and supportive information types.

Relations between information sources use and grade
H4 results: Participants who used information sources had better learning achievement than participants who did not use information sources; the difference is statistically significant regarding the reciprocal information type.
A chi-square test of independence was performed to examine the relation between information use and grade (objective learning achievement).
The relation between these variables was statistically significant, X 2 (2) = 16.58, p =.0003. Post hoc tests were conducted to evaluate differences among the use of information to the three grades scales (-100, 60, 10), separately for each information type. The results of these tests indicated a statistically significant difference between participant who experience information and those who didn't.
Statistically significant difference was found for the relation between the use of reciprocal information type and grade, X 2 (2) = 16.85, p =.0002. The relation between the use of the supportive information type and grade was also statistically significant, X 2 (2) = 7.67, p =.02. The relation between the use of the reflective information type and grade was statistically significant as well, X 2 (2) = 2.73, p =.26. Figure 4 shows grades for participants who did not use information (in dark gray), compared to participants who used any of the three information types (in light gray).  Figure 5 shows grades division by information type, for participants who used information sources. Using reciprocal information type led to the highest number of participants who answered correctly (grades 10 and 60).

Fig . 5 . Grades for participants w ho used information sources
Hypothesis H4 was partially accepted: participants who used information sources had better learning achievement than participants who filled the assignment without using information sources; the difference is significant when they experienced the supportive and reciprocal information types.

Discussion
The core assumption of the current research is that different types of information are associated with varying subjective and revealed values and with actual learning achievements in a problem-based task by software engineers.
The next sub-sections divide the results to pre-consumption subjective value and post-consumption revealed and UX values.

Pre-consumption
Since the value of information is inherently subjective, we measured the subjective value of each source type before usage. Subjective value for supportive information is lower than for the reflective information type, in contrast to our hypothesis. Moreover, at least half of the participants chose not to pay at all for supportive information type. The results show that participants expressed lower willingness-to-pay for the supportive information as well as low demand, indicated by the median in Table 4. Nelson (1970) claimed that before experiencing information, a consumer knows: (1) The price (zero or else); (2) Partial content such as title and a description; (3) Meta-data such as author, date, publication, reputation etc. In the present experiment, we let the participants set the price and found that metadata bears an influence on the Brought to you by | University of Haifa Authenticated Download Date | 8/2/18 7:50 AM subjective value assigned to information sources. Knowing that the information offered was a specification led to fewer purchases and a lower average price than transactions generated by knowing the information was a code example or tutorial.
Participants had the option to search for information on the Internet while participating in the experimental task. They were not limited to the given information sources. Despite having access to freely available information, they calculated their benefits from purchasing a direct, pre-selected reference instead of searching for help on the Web, as indicated by the choice of 55% of the participants. The limited selection prompted subjective value indicated by the willingness-to-pay. Lin et al. (2013) suggested that "free mentality", which is the consumer's expectation to receive information sources for free on the Internet, has been ironed into consumers' minds. This mentality reduces consumers' willingness to pay for online information. However, according to Lin et al. (2013), the more benefits users expect to receive from the online information sources, the less averse to paying they become.
Overall, pre-consumption value assessment indicates that metadata influences value perception and that providing a limited choice of pre-selected sources increases value perception.

Post-consumption
Post-consumption results show, as expected, that there is no correlation between subjective and revealed values of information, probably as a result of the subjective nature of value assignment and of the usage experience.
Participants assigned a higher UX value to reflective and reciprocal information types than to supportive information. Using the supportive information type, the consumer is fairly passive and engages in reading the specifications and instructions. Using the reciprocal information type, the consumer is aware of information hierarchy (best answers in the Q&A forum appear at the top) and can take part in the conversation. Using the reflective type, the consumer interacts with the interface, tries it, and experiences it. Being more involved or active elicits higher UX scores.
In the context of informal learning, learning is effective when it is woven in a real-life situation, within authentic context. Moreover, the environment in which the learners interact with the information has an essential impact on the way this information is evaluated (Ariely, 2000). Indeed, a positive correlation exists between the overall experience (UX) participants had while consuming the information, and their evaluation of the information post experiencing it. This emphasizes the impact of the experience on the revealed value and the importance of the environment in addition to content type.
Revealed information value is measured using a 1-5 rating while the UX value is measured by the AttrakDiff scale. The revealed value is measured directly after each question (source level), while the UX is measured in the end of the entire task which included five questions (system level). As we assumed, learners do not distinguish between the information and the system in which the information is presented. Learners evaluated their experience, including the content and the system. The overall UX value showed that participants valued the reflective and reciprocal information types more than the supportive type.
The positive correlation between learning achievement and revealed information value may seem obvious, but it is important to remember that it is relevant only to learners who experienced information. Participants who purchased information sources and experienced them valued these sources post experience. Participants who chose not to purchase information either valued their previous knowledge highly or did not perceive a high enough value from the information sources in order to suggest a winning bid. Either way, completing the assignment without information led to lower performance.
Reciprocal information is associated with higher learning achievement than reflective and supportive. Moreover, the average grade for participants who acquired supportive information was -10.41, meaning that although they used the supportive information type, they gave wrong answers. The contribution to grades by using the reciprocal information is shown in the results both through the correlation between grades and information types and when examining relations between grades when using and not using information sources.

Research Contribution
The innovation of this research is the study of three value perceptions: the subjective value before the use of information, the UX value, and the revealed information value after the use of information, in the context of informal software engineering learning using three types of information sources. Raban (2007) suggests that the value of information perceived by a user is dynamic and changes by experience. Our findings add the aspect of user experience and show that there is a positive correlation between UX value and revealed information value. Furthermore, no correlation between subjective and revealed information value emerged, which lends empirical support to the notion of experience.
The experience refers to the UX and to the knowledge gain. This division leads to an analysis in two separate fields: UX and experience in terms of value of information. This research contributes to both by showing a unique interplay between them.
The results apply to engineering education because UX design is a stage in most software development cycles, PBL is common, informal learning is prevalent and there is an abundance of information sources.
As for learning sources for software engineers, the present results indicate that learners had clear preference for the reflective and reciprocal information types over the supportive type. This subjective preference translated into assignment performance. We believe this is a clear indication to designers of learning environments to be open to specific user preferences and also to employ information sources for higher-order thinking. For example, interactions using features such as "try it yourself", and reciprocal information delivered using interaction between learners (e.g. forums) are more effective for informal learning than official specifications. Usage of reciprocal and reflective sources will improve the learning achievements as well as information revealed value due to better UX. In addition, providing a limited number of pre-selected sources enhances value perceptions and is a recommended strategy for developers of learning assignments.
This research focused on an informal software engineering learning process, using information sources available on the Internet. The research examined the interaction between the learner and the environment delivering the information. In addition, the research examined the correlation between the type of information and its perceived value in the eyes of the learner. Results contribute to empirical knowledge regarding factors which will allow professional communities to characterize and design effective learning processes.

Limitations
The current research is based on a laboratory experiment which is associated with three main limitations. First, analyses were performed per assignment question, not per participant, because most analyses refer to variables measured only in cases in which information was used and therefore the number of data points obtained for each participant is not uniform. Second, one may claim that there is no actual interaction between participants in the reciprocal information type. In practice, most StackOverflow users do not interact, but read and understand the information flow and hierarchy. When building the information sources for the questions in the assignment, we especially chose a forum discussion with deep and hierarchical information flow. Participants, as part of their engineering learning process, had to understand the discussion interactivity and hierarchy in order to select the correct solution. Third, although there was a hidden minimum price for each scaffold type, it was the same pre-defined price for all questions. Due to technical limitations, it was not a random price within a certain range. Since we analyzed the willingnessto-pay on a comparative basis, not by absolute value, this limitation is negligible.

Summary and Future Research
In a nutshell, this research found that formal information, which constitutes the supportive information type, is least appropriate for self-learners in software engineering education. If it is vital to expose learners Brought to you by | University of Haifa Authenticated Download Date | 8/2/18 7:50 AM to this kind of information due to professional requirements, then designers of learning environments should take care to create a high UX value. For example, they might consider re-packaging the formal specification into a question and answer format which corresponds to the reciprocal information type. Interestingly, usage of reciprocal information led to the highest learning achievements which suggests that conversational format is effective for learning. A possible explanation might be that learners identify with the askers and enjoy the answers as if they were written for them personally. Research is needed to explore this intuitive explanation and compare it with other plausible explanations. Conducting a comparative study across various areas of study could show whether preference for and effectiveness of reciprocal information is unique in software engineering informal learning or whether these findings apply broadly to other engineering areas or even other learning fields. In other words, are the preference for and effectiveness of types of information inherent in the information source itself or do they change between fields of study?
From a theoretical standpoint this study provides empirical support to subjective aspects of value perceptions. The findings show that subjective value and revealed value differ and are not mutually predicatable. Due to different measures used before and after experiencing information, future research should examine these variables on the same scale in order to determine whether experience enhances or reduces value perceptions.