Skip to main content
SpringerLink
Log in

Expert Judgment Quantification

  • Chapter
  • First Online:
Research and Technology Management in the Electricity Industry

Part of the book series: Green Energy and Technology ((GREEN))

Abstract

Expert judgments are used when there are no objective data is available. It is critical to solicit these judgments accurately for decision makers. This chapter reviews methods and issues around the expert judgment quantification.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 159.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 109.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Rohrbaugh J (1979) Improving the quality of group judgment: social judgment analysis and the Delphi technique. Organ Behav Hum Perform 24:73–92

    Google Scholar 

  2. Meyer MA, Booker JM (1991) Eliciting and analyzing expert judgment: a practical guide. Academic Press Limited, London

    Google Scholar 

  3. Keeney RL, von Winterfeldt D (1989) On the uses of expert judgment on complex technical problems. IEEE Trans Eng Manage 36:83–86

    Google Scholar 

  4. Durance P, Godet M (2010) Scenario building: uses and abuses. Technol Forecast Soc Chang 77:1488–1492

    Google Scholar 

  5. Bradfield R, Wright G, Burt G, Cairns G, Van Der Heijden K (2005) The origins and evolution of scenario techniques in long range business planning. Futures 37:795–812

    Google Scholar 

  6. Cooke RM (1991) Experts in uncertainty: opinion and subjective probability in science. Oxford University Press, New York

    Google Scholar 

  7. Simola K, Mengolini A, Bolado-Lavin R (2005) Formal expert judgment: an overview. European Commission, Directorate General Joint Research Centre (DG JRC), Institute for Energy. http://ie.jrc.ec.europa.eu/publications/scientific_publications/2005/EUR21772EN.pdf. Accessed 18 March 2011

  8. Börjeson L, Höjer M, Dreborg K-H, Ekvall T, Finnveden G (2006) Scenario types and techniques: towards a user’s guide. Futures 38:723–739

    Google Scholar 

  9. Amer M, Daim TU (2010) Application of technology roadmaps for renewable energy sector. Technol Forecast Soc Chang 77:1355–1370

    Google Scholar 

  10. Gerdsri N (2005) An analytical approach on building a technology development envelope (TDE) for roadmapping of emerging technologies. Ph.D. Dissertation, Systems Science: Engineering Management, Portland State University, Portland

    Google Scholar 

  11. Gerdsri P (2009) A systematic approach to developing national technology policy and strategy for emerging technologies. Ph.D. Dissertation, Engineering and Technology Management (ETM) Department, Portland State University, Portland

    Google Scholar 

  12. Schwartz P (1996) The Art of the Long View: Planning for the Future in an Uncertain World. Currency Doubleday, New York

    Google Scholar 

  13. Chermack TJ, Lynham SA, Ruona WEA (2001) A review of scenario planning literature. Futures Res Q 17:7–31

    Google Scholar 

  14. Kahn H, Wiener AJ (1967) The year 2000: a framework for speculation on the next thirty-three years. The Macmillan, New York

    Google Scholar 

  15. Varum CA, Melo C (2007) Strategic planning in an uncertain business environment: the diffusion of scenario planning. In: Conference Factores de Competitividade, Competitiveness Factors: A Portuguese Perspective, Aveiro

    Google Scholar 

  16. Mietzner D, Reger G (2005) Advantages and disadvantages of scenario approaches for strategic foresight. Int J Technol Intell Planning 1:220–239

    Google Scholar 

  17. Varum CA, Melo C (2010) Directions in scenario planning literature: a review of the past decades. Futures 42:355–369

    Google Scholar 

  18. Van Der Heijden K (1996) Scenarios: the art of strategic conversation. Wiley, Chichester

    Google Scholar 

  19. Rigby D, Bilodeau B (2007) Selecting management tools wisely. Harv Bus Rev 85:20–22

    Google Scholar 

  20. Porter AL, Roper AT, Mason TW, Rossini FA, Banks J (1991) Forecasting and management of technology, 1st edn. Wiley, New York

    Google Scholar 

  21. Saritas O, Oner MA (2004) Systemic analysis of UK foresight results: joint application of integrated management model and roadmapping. Technol Forecast Soc Chang 71:27–65

    Google Scholar 

  22. Chermack TJ, Lynham SA, van der Merwe L (2006) Exploring the relationship between scenario planning and perceptions of learning organization characteristics. Futures 38:767–777

    Google Scholar 

  23. Schoemaker PJH, van der Heijden CAJM (1992) Integrating scenarios into strategic planning at royal Dutch/Shell. Strategy Leadersh 20:41–46

    Google Scholar 

  24. Joseph CF (2000) Scenario planning. Technol Forecast Soc Chang 65:115–123

    Google Scholar 

  25. Hiltunen E (2009) Scenarios: process and outcome. J Futures Stud 13:151–152

    Google Scholar 

  26. Linneman RE, Klein HE (1979) The use of multiple scenarios by U.S. industrial companies. Long Range Plan 12:83–90

    Google Scholar 

  27. Linneman RE, Klein HE (1983) The use of multiple scenarios by U.S. industrial companies: a comparison study, 1977–1981. Long Range Plan 16:94–101

    Google Scholar 

  28. Amer M, Daim TU, Jetter A (2013) A review of scenario planning. Futures 46:23–40

    Google Scholar 

  29. Blomgren H, Jonsson P, Lagergren F (2011) Getting back to scenario planning: strategic action in the future of energy Europe. In: 8th international conference on the European energy market (EEM), Zagreb, pp 792–801

    Google Scholar 

  30. Silberglitt R, Hove A, Shulman P (2003) Analysis of the US energy scenarios: meta-scenarios, pathways, and policy implications. Technol Forecast Soc Chang 70:297–315

    Google Scholar 

  31. Czaplicka-Kolarz K, Stanczyk K, Kapusta K (2009) Technology foresight for a vision of energy sector development in Poland till 2030. Delphi survey as an element of technology foresighting. Technol Forecast Soc Chang 76:327–338

    Google Scholar 

  32. Di W, Rui N, Hai-ying S (2011) Scenario analysis of China’s primary energy demand and CO2 emissions based on IPAT model. Energy Procedia 5:365–369

    Google Scholar 

  33. Keles D, Möst D, Fichtner W (2011) The development of the German energy market until 2030—a critical survey of selected scenarios. Energy Policy 39:812–825

    Google Scholar 

  34. Zhou N, Lin J (2008) The reality and future scenarios of commercial building energy consumption in China. Energy Build 40:2121–2127

    Google Scholar 

  35. Winebrake JJ, Creswick BP (2003) The future of hydrogen fueling systems for transportation: an application of perspective-based scenario analysis using the analytic hierarchy process. Technol Forecast Soc Chang 70:359–384

    Google Scholar 

  36. Sørensen B, Hauge Petersen A, Juhl C, Ravn H, Søndergren C, Simonsen P, Jørgensen K, Henrik Nielsen L, Larsen HV, Erik Morthorst P, Schleisner L, Sørensen F, Engberg Pedersen T (2004) Hydrogen as an energy carrier: scenarios for future use of hydrogen in the Danish energy system. Int J Hydrogen Energy 29:23–32

    Google Scholar 

  37. Wietschel M, Hasenauer U, de Groot A (2006) Development of European hydrogen infrastructure scenarios–CO2 reduction potential and infrastructure investment. Energy Policy 34:1284–1298

    Google Scholar 

  38. Contaldi M, Gracceva F, Mattucci A (2008) Hydrogen perspectives in Italy: analysis of possible deployment scenarios. Int J Hydrogen Energy 33:1630–1642

    Google Scholar 

  39. Antoine B, Goran K, Neven D (2008) Energy scenarios for Malta. Int J Hydrogen Energy 33:4235–4246

    Google Scholar 

  40. Weisser D (2004) Costing electricity supply scenarios: a case study of promoting renewable energy technologies on Rodriguez, Mauritius. Renewable Energy 29:1319–1347

    Google Scholar 

  41. Chen T-Y, Yu OS, Hsu GJY, Hsu F-M, Sung W-N (2009) Renewable energy technology portfolio planning with scenario analysis: a case study for Taiwan. Energy Policy 37:2900–2906

    Google Scholar 

  42. Madlener R, Kowalski K, Stagl S (2007) New ways for the integrated appraisal of national energy scenarios: the case of renewable energy use in Austria. Energy Policy 35:6060–6074

    Google Scholar 

  43. Varho V, Tapio P (2005) Wind power in Finland up to the year 2025—‘soft’ scenarios based on expert views. Energy Policy 33:1930–1947

    Google Scholar 

  44. Morales JM, Mínguez R, Conejo AJ (2010) A methodology to generate statistically dependent wind speed scenarios. Appl Energy 87:843–855

    Google Scholar 

  45. Rowe G, Wright G (1999) The Delphi technique as a forecasting tool: issues and analysis. Int J Forecast 15:353–375

    Google Scholar 

  46. Raubitschek R (1988) Multiple scenario analysis and business planning. In: Lamb R, Shrivastava P (eds) Advances in strategic management, vol 5. JAI Press Inc., London

    Google Scholar 

  47. Linstone HA, Turoff M (1975) The Delphi method: techniques and applications. Addison-Wesley, Reading

    MATH  Google Scholar 

  48. Martino JP (1983) Technological forecasting for decision making, 2nd edn. North-Holland, New York

    Google Scholar 

  49. Rowe G, Wright G, Bolger F (1991) Delphi: a reevaluation of research and theory. Technol Forecast Soc Chang 39:235–251

    Google Scholar 

  50. Erffmeyer RC, Erffmeyer ES, Lane IM (1986) The Delphi technique: an empirical evaluation of the optimal number of rounds. Group and Organ Manage 11:120–128

    Google Scholar 

  51. Lindeman CA (1975) Delphi survey of priorities in clinical nursing research. Nurs Res 24:434–441

    Google Scholar 

  52. Torrance EP (1957) Group decision-making and disagreement. Soc Forces 35:314–318

    Google Scholar 

  53. Powell C (2003) The Delphi technique: myths and realities. J Adv Nurs 41:376–382

    Google Scholar 

  54. Woudenberg F (1991) An evaluation of Delphi. Technol Forecast Soc Chang 40:131–150

    Google Scholar 

  55. Jairath N, Weinstein J (1994) The Delphi methodology: a useful administrative approach. Can J Nurs Adm 7:29–42

    Google Scholar 

  56. Linstone HA, Turoff M (2011) Delphi: a brief look backward and forward. Technol Forecast Soc Chang 78:1712–1719

    Google Scholar 

  57. Chaffin WW, Talley WK (1980) Individual stability in Delphi studies. Technol Forecast Soc Chang 16:67–73

    Google Scholar 

  58. Gordon T, Pease A (2006) RT Delphi: an efficient, “round-less” almost real time Delphi method. Technol Forecast Soc Chang 73:321–333

    Google Scholar 

  59. Celiktas MS, Kocar G (2010) From potential forecast to foresight of Turkey’s renewable energy with Delphi approach. Energy 35:1973–1980

    Google Scholar 

  60. Steinert M (2009) A dissensus based online Delphi approach: an explorative research tool. Technol Forecast Soc Chang 76:291–300

    Google Scholar 

  61. Geist MR (2010) Using the Delphi method to engage stakeholders: a comparison of two studies. Eval Program Plann 33:147–154

    Google Scholar 

  62. Rauch W (1979) The decision Delphi. Technol Forecast Soc Chang 15:159–169

    Google Scholar 

  63. Turoff M (1970) The design of a policy Delphi. Technol Forecast Soc Chang 2:149–171

    Google Scholar 

  64. Delbecq AL, Van de Ven AH, Gustafson DH (1975) Group techniques for program planning. A guide to nominal group and Delphi processes. Scott, Foresman and Company, Glenview

    Google Scholar 

  65. Murphy M, Black N, Lamping D, McKee CM, Sanderson CFB, Askham J (1998) Consensus development methods, and their use in clinical guideline development. Health Technol Assess 2:1–65

    Google Scholar 

  66. Pill J (1971) The Delphi method: substance, context, a critique and an annotated bibliography. Socio-Econ Plan Sci 5:57–71

    Google Scholar 

  67. Graham B, Regehr G, Wright JG (2003) Delphi as a method to establish consensus for diagnostic criteria. J Clin Epidemiol 56:1150–1156

    Google Scholar 

  68. Brockhoff K (1975) The performance of forecasting groups in computer dialogue and face to face discussions. In: Linstone HA, Turoff M (eds) The Delphi method: techniques and applications. Addison-Wesley, Reading

    Google Scholar 

  69. Riggs WE (1983) The Delphi technique: an experimental evaluation. Technol Forecast Soc Chang 23:89–94

    Google Scholar 

  70. Larreche JC, Moinpour R (1983) Managerial judgment in marketing: the concept of expertise. J Mark Res 20:110–121

    Google Scholar 

  71. Coates JF (1975) In defense of Delphi: a review of Delphi assessment, expert opinion, forecasting, and group process by H. Sackman. Technol Forecast Soc Chang 7:193–194

    Google Scholar 

  72. Sackman H (1974) Delphi critique: expert opinion, forecasting, and group process. Lexington Books, Lexington

    Google Scholar 

  73. Martino JP (2003) A review of selected recent advances in technological forecasting. Technol Forecast Soc Chang 70:719–733

    Google Scholar 

  74. Breiner S, Cuhls K, Grupp H (1994) Technology foresight using a Delphi approach: a Japanese German study. R&D Manage 24:141–153

    Google Scholar 

  75. Biloslavo R, Dolinek S (2008) Scenario planning for climate strategies development by integrating group Delphi, AHP and dynamic fuzzy cognitive maps. In: Portland International Center for Management of Engineering and Technology (PICMET), Cape Town, South Africa, pp 1103–1111

    Google Scholar 

  76. Huss WR, Honton EJ (1987) Scenario planning—what style should you use? Long Range Plan 20:21–29

    Google Scholar 

  77. Bengisu M, Nekhili R (2006) Forecasting emerging technologies with the aid of science and technology databases. Technol Forecast Soc Chang 73:835–844

    Google Scholar 

  78. Gupta UG, Clarke RE (1996) Theory and applications of the Delphi technique: a bibliography (1975–1994). Technol Forecast Soc Chang 53:185–211

    Google Scholar 

  79. Iniyan S, Sumathy K (2000) An optimal renewable energy model for various end-uses. Energy 25:563–575

    Google Scholar 

  80. Iniyan S, Sumathy K (2003) The application of a Delphi technique in the linear programming optimization of future renewable energy options for India. Biomass Bioenergy 24:39–50

    Google Scholar 

  81. Pätäri S (2010) Industry- and company-level factors influencing the development of the forest energy business—insights from a Delphi study. Technol Forecast Soc Chang 77:94–109

    Google Scholar 

  82. Rikkonen P, Tapio P (2009) Future prospects of alternative agro-based bioenergy use in Finland–Constructing scenarios with quantitative and qualitative Delphi data. Technol Forecast Soc Chang 76:978–990

    Google Scholar 

  83. Ronde P (2003) Delphi analysis of national specificities in selected innovative areas in Germany and France. Technol Forecast Soc Chang 70:419–448

    Google Scholar 

  84. Sharma DP, Nair PSC, Balasubramanian R (2003) Analytical search of problems and prospects of power sector through Delphi study: case study of Kerala state, India. Energy Policy 31:1245–1255

    Google Scholar 

  85. Suganthi L, Williams A (2000) Renewable energy in India—a modelling study for 2020–2021. Energy Policy 28:1095–1109

    Google Scholar 

  86. Utgikar VP, Scott JP (2006) Energy forecasting: predictions, reality and analysis of causes of error. Energy Policy 34:3087–3092

    Google Scholar 

  87. Blind K, Cuhls K, Grupp H (2001) Personal attitudes in the assessment of the future of science and technology: a factor analysis approach. Technol Forecast Soc Chang 68:131–149

    Google Scholar 

  88. Cuhls K, Kuwahara T (1994) Outlook of Japanese and German future technology: comparing technology forecast surveys. Phisica-Verlag, Heidelberg

    Google Scholar 

  89. Amer M, Daim T (2012) Technology and science policies in transitional economy: a case of Turkey. Sci Technol Soc 17:297–321

    Google Scholar 

  90. Gerdsri P Kocaoglu D (2009) A systematic approach to developing national technology policy and strategy for emerging technologies: a case study of nanotechnology for Thailand’s agriculture industry. In: Portland International Center for Management of Engineering and Technology (PICMET), Portland, pp 447–461

    Google Scholar 

  91. Chakravarti AK, Vasanta B, Krishnan ASA, Dubash RK (1998) Modified Delphi methodology for technology forecasting case study of electronics and information technology in India. Technol Forecast Soc Chang 58:155–165

    Google Scholar 

  92. Shin T (1998) Using Delphi for a long-range technology forecasting, and assessing directions of future R&D activities the Korean exercise. Technol Forecast Soc Chang 58:125–154

    Google Scholar 

  93. Tichy G (2004) The over-optimism among experts in assessment and foresight. Technol Forecast Soc Chang 71:341–363

    Google Scholar 

  94. Daim T, Yates D, Peng Y, Jimenez B (2009) Technology assessment for clean energy technologies: the case of the Pacific northwest. Technol Soc 31:232–243

    Google Scholar 

  95. Saaty TL (2003) Decision-making with the AHP: why is the principal eigenvector necessary. Eur J Oper Res 145:85–91

    MathSciNet  MATH  Google Scholar 

  96. Wang J–J, Jing Y–Y, Zhang C-F, Zhao J-H (2009) Review on multi-criteria decision analysis aid in sustainable energy decision-making. Renew Sustain Energy Rev 13:2263–2278

    Google Scholar 

  97. Salo A, Gustafsson T, Ramanathan R (2003) Multicriteria methods for technology foresight. J Forecast 22:235–255

    Google Scholar 

  98. Elkarmi F, Mustafa I (1993) Increasing the utilization of solar energy technologies (SET) in Jordan analytic hierarchy process. Energy Policy 21:978–982

    Google Scholar 

  99. Forman EH, Gass SI (2001) The analytic hierarchy process—an exposition. Operations Research 49:469–486

    MathSciNet  MATH  Google Scholar 

  100. Saaty TL (1980) The analytic hierarchy process: planning, priority setting, resource allocation. McGraw Hill International, New York

    Google Scholar 

  101. Gerdsri N, Kocaoglu DF (2007) Applying the analytic hierarchy process (AHP) to build a strategic framework for technology roadmapping. Math Comput Model 46:1071–1080

    Google Scholar 

  102. Kocaoglu DF (1983) A participative approach to program evaluation. IEEE Trans Eng Manage 30:112–118

    Google Scholar 

  103. Kablan MM (2004) Decision support for energy conservation promotion: an analytic hierarchy process approach. Energy Policy 32:1151–1158

    Google Scholar 

  104. Ahsan MK, Bartlema J (2004) Monitoring healthcare performance by analytic hierarchy process: a developing-country perspective. Int Trans Oper Res 11:465–478

    MATH  Google Scholar 

  105. Yates D, Jimenez BT, Peng Y (2007) Portland general electric (PGE): clean power generation wind project in biglow canyon boardman coal plant. In: PICMET, Portland, pp 2530–2549

    Google Scholar 

  106. Akash BA, Mamlook R, Mohsen MS (1999) Multi-criteria selection of electric power plants using analytical hierarchy process. Electric Power Syst Res 52:29–35

    Google Scholar 

  107. Lee SK, Yoon YJ, Kim JW (2007) A study on making a long-term improvement in the national energy efficiency and GHG control plans by the AHP approach. Energy Policy 35:2862–2868

    Google Scholar 

  108. Lee SK, Mogi G, Kim JW (2008) The competitiveness of Korea as a developer of hydrogen energy technology: the AHP approach. Energy Policy 36:1284–1291

    Google Scholar 

  109. Lee SK, Mogi G, Kim JW, Gim BJ (2008) A fuzzy analytic hierarchy process approach for assessing national competitiveness in the hydrogen technology sector. Int J Hydrogen Energy 33:6840–6848

    Google Scholar 

  110. Lee AHI, Chen HH, Kang H-Y (2008) Multi-criteria decision making on strategic selection of wind farms. Renewable Energy 34:120–126

    Google Scholar 

  111. Aras H, Erdogmus S, Koc E (2004) Multi-criteria selection for a wind observation station location using analytic hierarchy process. Renewable Energy 29:1383–1392

    Google Scholar 

  112. Jaber JO, Jaber QM, Sawalha SA, Mohsen MS (2008) Evaluation of conventional and renewable energy sources for space heating in the household sector. Renew Sustain Energy Rev 12:278–289

    Google Scholar 

  113. Lee TL, Lin HM, Jeng DS, Hsu TW (2008) Application of fuzzy analytic hierarchy process to assess the potential of offshore wind energy in Taiwan. In: Proceedings of the international offshore and polar engineering conference. Vancouver, pp 461–465

    Google Scholar 

  114. Pilavachi PA, Chatzipanagi AI, Spyropoulou AI (2009) Evaluation of hydrogen production methods using the analytic hierarchy process. Int J Hydrogen Energy 34:5294–5303

    Google Scholar 

  115. Tsoutsos T, Drandaki M, Frantzeskaki N, Iosifidis E, Kiosses I (2009) Sustainable energy planning by using multi-criteria analysis application in the island of Crete. Energy Policy 37:1587–1600

    Google Scholar 

  116. Wijayatunga PDC, Siriwardena K, Fernando WJLS, Shrestha RM, Attalage RA (2006) Strategies to overcome barriers for cleaner generation technologies in small developing power systems: Sri Lanka case study. Energy Convers Manage 47:1179–1191

    Google Scholar 

  117. Chatzimouratidis AI, Pilavachi PA (2008) Multicriteria evaluation of power plants impact on the living standard using the analytic hierarchy process. Energy Policy 36:1074–1089

    Google Scholar 

  118. Chatzimouratidis AI, Pilavachi PA (2009) Technological, economic and sustainability evaluation of power plants using the analytic hierarchy process. Energy Policy 37:778–787

    Google Scholar 

  119. Clarke RR (1997) Validation and legitimation of an analytic hierarchy approach to integrated resource planning for electric utilities. In: Energy conversion engineering conference (IECEC-97), Honolulu, pp 2197–2201

    Google Scholar 

  120. Afgan NH, Carvalho MG (2002) Multi-criteria assessment of new and renewable energy power plants. Energy 27:739–755

    Google Scholar 

  121. Amer M, Daim TU (2011) Selection of renewable energy technologies for a developing country: a case study of Pakistan. Energy Sustain Dev 15:420–435

    Google Scholar 

  122. Millett SM, Honton EJ (1991) A manager’s guide to technology forecasting and strategy analysis methods. Battelle Press, Columbus

    Google Scholar 

  123. Rohrbaugh J (1981) Improving the quality of group judgment: social judgment analysis and the nominal group technique. Organ Behav Hum Perform 28:272–288

    Google Scholar 

  124. Herbert TT, Yost EB (1979) A comparison of decision quality under nominal and interacting consensus group formats: the case of the structured problem. Decis Sci 10:358–370

    Google Scholar 

  125. Islam R (2001) Modification of the nominal group technique by using the analytic hierarchy process. In: Multiple Criteria Decision Making in the New Millennium, vol 507, pp 294–303

    Google Scholar 

  126. Henderson NR (2009) Managing moderator stress: take a deep breath. You can do this! Mark Res 21:28–29

    Google Scholar 

  127. Sutton RI, Hargadon A (1996) Brainstorming groups in context: effectiveness in a product design firm. Adm Sci Q 41:685–718

    Google Scholar 

  128. Miles I, Keenan M (2002) Practical guide to regional foresight in the United Kingdom. Directorate-General for Research, European Commission. http://cordis.europa.eu/foresight/cgrf.htm. Accessed 14 March 2011

  129. Keeney RL, von Winterfeldt D (1991) Eliciting probabilities from experts in complex technical problems. IEEE Trans Eng Manage 38:191–201

    Google Scholar 

  130. Cojazzi G, Fogli D, Grassini G, De Gelder P, Gryffroy D, Bolado R, Hofer E, Virolainen R, Coe IM, Bassanelli A, Puga J, Papazoglou I, Zuchuat O, Cazzoli E, Eyink J, Guida G, Pinola L, Pulkkinen U, Simola K, von Winterfeldt D, Valeri A (2001) Benchmark exercise on expert judgment techniques in PSA Level 2. Nucl Eng Des 209:211–221

    Google Scholar 

  131. Cojazzi G, Fogli D (2000) Benchmark exercise on expert judgment techniques in PSA level 2, extended final report. http://ie.jrc.ec.europa.eu/publications/scientific_publications/2005/EUR21772EN.pdf

  132. Meyer MA, Booker JM (1991) Selecting and motivating the experts. In: Eliciting and analyzing expert judgment: a practical guide, pp 85–98, 1st edn. Academic Press Limited, London

    Google Scholar 

  133. Camerer CF, Johnson EJ (1991) The process-performance paradox in expert judgment. In: Ericsson KA, Smith J (eds) Toward a general theory of expertise: prospects and limits, 1st edn. Cambridge University Press, Cambridge, pp 195–217

    Google Scholar 

  134. McGraw KL, Harbison-Briggs K (1989) Knowledge acquisition: principles and guidelines. Prentice-Hall International, New Jersey

    Google Scholar 

  135. Landeta J (2006) Current validity of the Delphi method in social sciences. Technol Forecast Soc Chang 73:467–482

    Google Scholar 

  136. Jetter AJ (2006) Elicitation-extracting knowledge from experts. In: Jetter AJ, Kraaijenbrink J, Schroder H-H, Wijnhoven F (eds) Knowledge integration: the practice of knowledge management in small and medium enterprises, pp 65–76. Physica-Verlag, Heidelberg

    Google Scholar 

  137. Schaller RR (2004) Technology innovation in the semiconductor industry: a case study of the international technology roadmap for semiconductors (ITRS) Ph.D. Dissertation, School of Public Policy, George Mason University, Fairfax

    Google Scholar 

  138. Shanteau J (1992) The psychology of experts: an alternative view. In: Wright, G. Bolger F (eds) Expertise and decision support, Plenum Press, pp 11–23. New York

    Google Scholar 

  139. Ascher W (1978) Forecasting: An Appraisal for Policymakers and Planners. Johns Hopkins University Press, Baltimore

    Google Scholar 

  140. Mitchell VW (1991) The Delphi technique: an exposition and application. Technol Anal Strat Manage 3:333–358

    Google Scholar 

  141. Okoli C, Pawlowsk SD (2004) The Delphi method as a research tool: an example, design considerations and applications. Inf Manage 42:15–29

    Google Scholar 

  142. Daim TU (1998) Technology evaluation and acquisition strategies and their implications in the U.S. electronics manufacturing industry. Ph.D. Dissertation, System Science: Engineering and Technology Management, Portland State University, Portland

    Google Scholar 

  143. Hill KQ, Fowles J (1975) The methodological worth of the Delphi forecasting technique. Technol Forecast Soc Chang 7:179–192

    Google Scholar 

  144. Cooke RM, Goossens LHJ (2000) Procedures guide for structured expert judgement. http://ie.jrc.ec.europa.eu/publications/scientific_publications/2005/EUR21772EN.pdf. Accessed 12 May 2011

  145. Dajani JS, Sincoff MZ, Talley WK (1979) Stability and agreement criteria for the termination of Delphi studies. Technol Forecast Soc Chang 13:83–90

    Google Scholar 

  146. Kastein MR, Jacobs M, van der Hell RH, Luttik K, Touw-Otten FWMM (1993) Delphi, the issue of reliability: a qualitative Delphi study in primary health care in the Netherlands. Technol Forecast Soc Chang 44:315–323

    Google Scholar 

  147. Goodman LA, Kruskal WH (1959) Measures of association for cross classifications. II: further discussion and references. J Am Stat Assoc 54:123–163

    MATH  Google Scholar 

  148. Mumpower JL, Stewart TR (1996) Expert judgement and expert disagreement. Thinking Reasoning 2:191–211

    Google Scholar 

  149. Bland JM, Altman DG (1997) Statistics notes: Cronbach’s alpha. Br Med J 314:572

    Google Scholar 

  150. Terrados J, Almonacid G, Pérez-Higueras P (2009) Proposal for a combined methodology for renewable energy planning. Application to a Spanish region. Renew Sustain Energy Rev 13:2022–2030

    Google Scholar 

  151. Shrout PE, Fleiss JL (1979) Intraclass correlations: uses in assessing rater reliability. Psychol Bull 86:420–428

    Google Scholar 

  152. Wuensch KL (2007) Inter-rater agreement. http://www.ecu.edu/psyc/

  153. Coates JF (1975) In defense of Delphi: a review of Delphi assessment: expert opinion, forecasting, and group process by H. Sackman. Technol Forecast Soc Chang 7:193–194

    Google Scholar 

  154. Kandasamy WBV, Smarandache F (2003) Fuzzy cognitive maps and neutrosophic cognitive maps. Indian institute of technology, Chennai

    MATH  Google Scholar 

  155. Taber R (1991) Knowledge processing with fuzzy cognitive maps. Expert Syst Appl 2:83–87

    Google Scholar 

  156. Taber W, Siegel M (1987) Estimation of experts’ weights using fuzzy cognitive maps. In: IEEE International Conference on Neural Networks, pp 319–326

    Google Scholar 

  157. Taber R, Yager RR, Helgason CM (2007) Quantization effects on the equilibrium behavior of combined fuzzy cognitive maps. Int J Intell Syst 22:181

    MATH  Google Scholar 

  158. Hans-Horst S, Jetter AJM (2003) Integrating market and technological knowledge in the fuzzy front end: an FCM-based action support system. Int J Technol Manage 26:517–539

    Google Scholar 

  159. Jetter AJM (2003) Educating the guess: strategies, concepts, and tools for the fuzzy front end of product development. In: Portland International Center for Management of Engineering and Technology (PICMET), Portland, pp 261–273

    Google Scholar 

  160. Amer M, Jetter AJ, Daim TU (2011) Development of fuzzy cognitive map (FCM) based scenarios for wind energy. Int J Energy Sect Manage 5:564–584

    Google Scholar 

  161. Babbie ER (2007) The practice of social research, 12th edn. Wadsworth, Belmont

    Google Scholar 

  162. Ono R, Wedemeyer DJ (1994) Assessing the validity of the Delphi technique. Futures 26:289–304

    Google Scholar 

  163. Martino JP (1970) The consistency of Delphi forecasts. Futurist 4:63–64

    Google Scholar 

  164. Keeney S, Hasson F, McKenna HP (2001) A critical review of the Delphi technique as a research methodology for nursing. Int J Nurs Stud 38:195–200

    Google Scholar 

  165. Engels TCE, Powell Kennedy H (2007) Enhancing a Delphi study on family-focused prevention. Technol Forecast Soc Chang 74:433–451

    Google Scholar 

  166. van Zolingen SJ, Klaassen CA (2003) Selection processes in a Delphi study about key qualifications in senior secondary vocational education. Technol Forecast Soc Chang 70:317–340

    Google Scholar 

  167. Jolson MA, Rossow GL (1971) The Delphi process in marketing decision making. J Mark Res 8:443–448

    Google Scholar 

  168. Brown BB, Helmer O (1964) Improving the reliability of estimates obtained from a consensus of experts, pp 1–13. http://www.rand.org/pubs/papers/P2986

  169. Dalkey NC, Brown BB, Cochran SW (1970) The Delphi Method IV: effect of percentile feedback and feed-in of relevant facts. http://www.rand.org/pubs/research_memoranda/RM6118

  170. Landeta J, Matey J, Ruíz V, Galter J (2008) Results of a Delphi survey in drawing up the input-output tables for Catalonia. Technol Forecast Soc Chang 75:32–56

    Google Scholar 

  171. Chambers JC, Mullick SK, Smith DD (1971) How to choose the right forecasting technique. Harv Bus Rev 49:45–74

    Google Scholar 

  172. Levary RR, Han D (1995) Choosing a technological forecasting method, Indus Manag 37:14–18

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Portland State University, Portland, USA

    Muhammad Amer & Tugrul Daim

Authors
  1. Muhammad Amer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Tugrul Daim
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tugrul Daim .

Editor information

Editors and Affiliations

  1. , Engineering and Technology Management, Portland State University, SW Fourth, Portland, 97201, USA

    Tugrul Daim

  2. Bonneville Power Administration, NE 11th Ave 905, Portland, 97207, Oregon, USA

    Terry Oliver

  3. Bonneville Power Administration, NE 11th Ave 905, Portland, 97232, Oregon, USA

    Jisun Kim

Rights and permissions

Reprints and permissions

Copyright information

© 2013 Springer-Verlag London

About this chapter

Cite this chapter

Amer, M., Daim, T. (2013). Expert Judgment Quantification. In: Daim, T., Oliver, T., Kim, J. (eds) Research and Technology Management in the Electricity Industry. Green Energy and Technology. Springer, London. https://doi.org/10.1007/978-1-4471-5097-8_3

Download citation

  • DOI: https://doi.org/10.1007/978-1-4471-5097-8_3

  • Published:

  • Publisher Name: Springer, London

  • Print ISBN: 978-1-4471-5096-1

  • Online ISBN: 978-1-4471-5097-8

  • eBook Packages: EnergyEnergy (R0)

Publish with us

Policies and ethics

Search

Navigation