Expert assessment method in foresight studies

Purpose of the study. The main goal of this research is to identify key aspects of expert assessments and offer high-quality recommendations for their improvement.

Materials and methods. Foresight is built on the basis of expert assessment method, includes: active formation of the image of the future instead of its probabilistic prediction, focus on identifying key development priorities, participation in the study of key stakeholders, the relationship with the management decision-making process. The methods of analysis used in the work suggest methods of theoretical research in the form of analysis and modeling. In the course of the research, the following tasks were solved: firstly, the Expert model was formed based on the necessary and sufficient criteria for selecting respondents to the foresight study; secondly, the main characteristics of the Delphi method for consensus decisions in expert groups were identified. The research work considers various sources of information, which became the basis for the further development of the Expert model, based on an integrated approach based on the statistical, sociological and economic fields of science.

Results. On the basis of bibliometric analysis, important criteria for the selection of experts were highlighted: a practical component, a theoretical component, a creative component, an assessment of belonging to a field of study, an assessment of work in a study, an adjustment of results. The relevance of the selection of an expert greatly influences the result of the foresight, therefore, there is a need for a balanced selection of respondents to the study. Foresight studies have a distinctive feature from other areas in that the result is the achievement of consensus between experts in the subject area. Decision makers are drawn from three areas of activity business, government, science. In this regard, criteria for the selection of respondents were formed, which imply the necessary and sufficient conditions. The necessary criteria are understood as such parameters, without which the characteristics of the expert do not allow the respondent to be an expert for this study. A sufficient condition for the participation of the decision maker in the foresight analysis implies such selection criteria, which are complementary characteristics of the expert, which do not need to prove that the expert is a suitable expert for a specific study. As a result, four necessary criteria for the selection of experts for groups were identified, as well as individual sufficient criteria for each group. The process of carrying out the Delphi method is considered, the advantages and disadvantages are determined, on the basis of which the resulting indicator is proposed the foresight research reliability index.

Conclusion. The method of selection of experts allows a comprehensive approach to the problem in the field of formation of expert groups based on the introduction of digital technologies that improves the qualitative characteristics of foresight research. The confidence index, as a result indicator, determines the objectivity of the study based on expert assessments.

1. Collopy F., Adya M., Armstrong J. S. Expert Systems for Forecasting. Principles of Forecasting. 2001; 285–300.

2. Shanteau J. How much information does an expert use? Is it relevant?. Acta Psychologica. 1992; 75–86.

3. Robert W. Blanning, Management Applications of Expert Systems. Elsevier Science Publishers B.V. (North-Holland). 1984; 311–316.

4. J. Scott Armstrong, J. Thomas Yokum Potential diffusion of expert systems in forecasting. Technological Forecasting and Social Change. 2001; 93–103.

5. Chernysheva T.YU. Hierarchical model of assessment and selection of experts // Management, computer engineering and informatics. Doklady TUSURa = Reports of TUSUR. 2009; 1 (19): 168–173. (In Russ.)

6. Maslennikov Ye.V. Possibilities of using expert knowledge as a source of organization development concepts .Vestnik Moskovskogo universiteta. 18. Sotsiologiya i politologiya = Bulletin of Moscow University. 18. Sociology and political science. 2017; 2: 229–250. (In Russ.)

7. Gorshkov M.K., Sheregi F.E. Prikladnaya sotsiologiya: metodologiya i metody: uchebnoye posobiye = Applied Sociology: Methodology and Methods: a training manual. 2nd ed. Moscow: “Center for Sociological Research”, Institute of Sociology RAS; 2012. 403 p. (In Russ.)

8. Lyaskovskiy V.L., Smirnov S.S., Pronin A.YU. Selection of experts for evaluating draft program documents. Kompetentnost’ = Competency. 2017; 4: 4–15. (In Russ.)

9. Larichev O. I., Morgoyev V. K. Problems, methods and systems for extracting expert knowledge. Avtomatika i telemekhanika = Automation and Telemechanics. 1991; 6: 3–27. (In Russ.)

10. Kukushkina S.N. The method of delphi in foresight projects. Forsayt = Foresight. 2007; 1(1): 68–72. (In Russ.)

11. Sidel’nikov YU.V. Sistemnyy analiz ekspertnogo prognozirovaniya = System analysis of expert forecasting. Moscow: Moscow Aviation Institute; 2007. 453 p. (In Russ.)

12. Gutsykova S.V. Metod ekspertnykh otsenok. Teoriya i praktika. = Method of expert assessments. Theory and practice. Moscow: Institute of Psychology of the Russian Academy of Sciences; 2011. 144 p. (In Russ.)

13. Karasev O.I., Kitayev A.Ye., Mironova I.I., Shinkarenko T.V. Foresight expert procedures: features of interaction with experts in long-term forecasting projects. Vestnik SpbGu = Bulletin of St. Petersburg State University. Sociology 2017; 10: 170–185. (In Russ.)

14. Murinovich A.A., Loginov M.P Foresight project as a basis for strategic interregional planning. Izvestiya UrGU = News of Ural State University. 2017; 1 (69): 101–116. (In Russ.)

15. Sidorov A.A. Issledovaniye sotsial’noekonomicheskikh i politicheskikh protsessov = The study of socio-economic and political processes. Tomsk: TUSUR; 2005. 266 p. (In Russ.)

16. Dalkey N., Helmer O. An experimental application of the Delphi method to the use of experts. Management Science. 1963; 9; 3: 458–467.

17. Sniezek J. A. A comparison of techniques for judgmental forecasting by groups with common information. Group and Organization Studies. 1990; 15(1): 5–19.

18. Spinelli T. The Delphi decision-making process. Journal of Psychology. 1983; 1(13): 73–80.

19. Boje D. M., Murnighan J. K. Group confidence pressures in iterative decisions. Management Science. 1982; 28: 1187–1196.

20. Fischer G. W. When oracles fail, a comparison of four procedures for aggregating subjective probability forecasts. Organizational Behavior and Human Performance. 1981; 28: 96–110.

21. Powell C. The Delphi technique: myths and realities. Methodological issue in nursing research. Blackwell Publishing Ltd. 2003: 376–382.

22. Okoli C., Pawlowski S.D. The Delphi method as a research tool: an example, design considerations and applications. Information & Management. 2004; 42: 15–29.

23. Orlov A.I. Organizatsionno-ekonomicheskoye modelirovaniye, uchebnik : v 3 ch. = Organizational and economic modeling, textbook: at 3 o’clock. Moscow: Publishing house of MGTU im. N.E. Bauman, 2009. 486 p. (In Russ.)

24. Rowe G., Wright G. The Delphi technique as a forecasting tool: issues and analysis. International Journal of Forecasting. 1999; 15: 353–375.

25. Hsu C., Brian A. The Delphi Technique: Making Sense Of Consensus. Practical assessment, research and evaluation. 2007; 12: 1–8.