Application-specific reasoning mechanisms (ASRMs) development is a rapidly growing domain of systems engineering. A demonstrative implementation of an active recommender system (ARS) was realized to support designing ASRMs and to circumvent procedural obstacles by providing context-sensitive recommendations. The specific problem for the research presented in this paper was the development of a synthetic validation agent (SVA) to simulate the decisional behaviour of designers and to generate data about the usefulness of the recommendations. The fact of the matter is that the need for the SVA was raised by the pandemic, which prevented involving groups of human designers in the recommendation testing process. The reported research had three practical goals: (i) development of the logical fundamentals for the SVA, (ii) computational implementation of the SVA, and (iii) application of the SVA in data generation for the evaluation of usefulness of recommendation. The SVA is based on a probabilistic decisional model that quantifies decisional options according to the assumed decisional tendencies. The three key concepts underlying the SVA are (i) decisional logic, (ii) decisional knowledge, and (iii) decisional probability. These together enable generation of reliable data about the decisional behaviours of human designers concerning the obtained recommendations. The completed tests proved the above assumption.
Asadollahi,S., Rafe,V., Rafeh,R. (2009). Towards automated software verification and validation. In: Proceedings of the 2009 International Conference on Computer Technology and Development, Vol. 1, IEEE, 206-210.
2.
Ayal,S., Hochman,G.U. (2009). Ignorance or integration: The cognitive processes underlying choice behavior. Journal of Behavioral Decision Making, 22(4), 455–474.
3.
Balke,T., Gilbert,N. (2014). How do agents make decisions? – A survey. Journal of Artificial Societies and Social Simulation, 17(4), 13–43.
4.
Burciu,A., Hapenciuc,C.V. (2010). Non-rational thinking in the decision-making process. In Proceedings of the European Conference on Intellectual Capital, 2, 152–160.
5.
Caire,G., Cossentino,M., Negri,A., Poggi,A., Turci,P. (2004). Multi-agent systems implementation and testing. In: Proceedings of the Agent Technology to Agent Implementation Symposium, 1-10.
6.
Choi,J., Choi,B. (2002). Test agent system. International Journal of Software Engineering and Knowledge Engineering, 12(03), 269–290.
7.
Chung,H.M., Silver,M.S. (1992). Rule-based expert systems and linear models: An empirical comparison of learning-by-examples methods. Decision Sciences, 23(3), 687–707.
8.
Creyer,E.H., Bettman,J.R., Payne,J.W. (1990). The impact of accuracy and effort feedback and goals on adaptive decision behavior. Journal of Behavioral Decision Making, 3(1), 1–16.
9.
Dastani,M., Hulstijn,J., van der Torre,L. (2001). BDI and QDT: A comparison based on classical decision theory. In: Proceedings of the AAAI Spring Symposium on Game Theoretic and Decision Theoretic Agents, 16-26.
10.
Dorer,K. (2004). Extended behavior networks for behavior selection in dynamic and continuous domains. In: Proceedings of the ECAI Workshop on Agents in Dynamic Domains, Valencia, Spain, 1-6.
11.
Engel,A. (2010) Verification, validation, and testing of engineered systems (Vol. 73). John Wiley &Sons, Hoboken, N.J.
12.
Friess,N., Crawford,H., Aycock,J. (2009).Amulti-agent approach to testing anti-spam software. In: Proceedings of the 20th International Workshop on Database and Expert Systems Application, IEEE, 38-42.
13.
Frolik,J., Abdelrahman,M., Kandasamy,P. (2001). A confidence-based approach to the self-validation, fusion and reconstruction of quasi-redundant sensor data. IEEE Transactions on Instrumentation and Measurement, 50(6), 1761–1769.
14.
Gibson,W., Powell-Evans,K. (1998). Validation fundamentals: How to, what to, when to validate. CRC Press. Boca Raton, 1-248.
15.
Grundy,J., Ding,G. (2002). Automatic validation of deployed J2EE components using aspects. In: Proceedings of the 17th International Conference on Automated Software Engineering, IEEE, 47-56.
16.
Grundy,J., Ding,G., Hosking,J. (2005). Deployed software component testing using dynamic validation agents. Journal of Systems and Software, 74(1), 5–14.
17.
Guerini,M., Moneta,A. (2017). A method for agent-based models validation. Journal of Economic Dynamics and Control, 82, 125–141.
18.
Huang,J., Wu,W., Zhang,Z., Chen,Y. (2020). A human decision-making behavior model for human-robot interaction in multi-robot systems. IEEE Access, 8, 197853–197862.
19.
Jager,W., Janssen,M. (2012). An updated conceptual framework for integrated modeling of human decision making: The Consumat II. In: Proceedings of the Workshop Complexity in the Real World@ECCS, 1-18.
20.
Jain,S., Patel,A. (2021). Semantic contextual reasoning to provide human behavior. Retrieved from https://arxiv.org/abs/2103.10694.
21.
Kant,J.D., Thiriot,S. (2006). Modeling one human decision maker with a multi-agent system: The CODAGE approach. In: Proceedings of the Fifth International Joint Conference on Autonomous Agents and Multiagent Systems, 50-57.
22.
Kim,C.N., Yang,K.H., Kim,J. (2008). Human decision-making behavior and modeling effects. Decision Support Systems, 45(3), 517–527.
23.
Klügl,F. (2008). A validation methodology for agent-based simulations. In: Proceedings of the Symposium on Applied Computing, ACM, 39-43.
24.
Kumar,A., GoyalV. (2012). EARCT - Environment for automatedrank based continuous agent testing. International Journal ofComputing, 11(3), 180–190.
25.
Kumaresen,P.P., Frasheri,M., Enoiu,E.P. (2020). Agent-based software testing:Adefinition and systematic mapping study. In: Proceedings of the 20th International Conference on Software Quality, Reliability and Security Companion, IEEE, 24-31.
26.
KwasniewskiL. (2009). On practical problems with verification and validation of computational models. Archives of Civil Engineering, 55(3), 323–346.
27.
Maes,P. (1989). The dynamics of action selection. In: Proceedings of the International Joint Conference on Artificial Intelligence, 991–997.
28.
Manzoor,U., Irfan,J., Nefti,S. (2011). Autonomous agents for testing and verification of softwares after deployment over network. In: Proceedings of the 2011 World Congress on Internet Security, IEEE, 36-41.
29.
Meng,Y., Qasem,S.N., Shokri,M. (2020). Dimension reduction of machine learning-based forecasting models employing principal component analysis. Mathematics, 8(8), 1233, pp. 1–15.
30.
Moghadam,M.H. (2019). Machine learning-assisted performance testing. In: Proceedings of the 27th Joint Meeting on European Software Engineering Conference and Symposium on the Foundations of Software Engineering, ACM, 1187-1189.
31.
Moghadam,M.H., Hamidi,G., Borg,M., Saadatmand,M., Bohlin,M., Lisper,B., Potena,P. (2021). Performance testing using a smart reinforcement learning-driven test agent. arXiv preprint arXiv:2104.12893.
32.
Monschein,D., Mazkatli,M., Heinrich,R., Koziolek,A. (2021). Enabling consistency between software artefacts for software adaption and evolution. In: Proceedings of the 18th International Conference on Software Architecture, IEEE, 1-12.
33.
Mudengudi,S.S., Kakkasageri,M.S. (2022) Cooperative agent-based location validation for vehicular clouds. In: IoT and Analytics for Sensor Networks. Springer, Singapore, 171–181.
34.
Mutanu,L., Kotonya,G. (2018). A self-learning approach for validation of runtime adaptation in service-oriented systems. Service Oriented Computing and Applications, 12(1), 11–24.
35.
Nguyen,C.D., Perini,A., Tonella,P. (2008). eCAT: A tool for automating test cases generation and execution in testing multi-agent systems (demo paper). In: Proceedings of AAMAS-08, 1669-1670.
Parker,A.M., Fischhoff,B. (2005). Decision-making competence: External validation through an individual-differences approach. Journal of Behavioral Decision Making, 18(1), 1–27.
38.
Petty,R.E., Briñol,P., Tormala,Z.L. (2002). Thought confidence as a determinant of persuasion: The self-validation hypothesis. Journal of Personality and Social Psychology, 82(5), 722–741.
39.
Raghavan,A., Baras,J.S. (2019). Non-commutative probability models in human decision making: Binary hypothesis testing. IFAC-PapersOnLine, 51(34), 47–52.
40.
Ramspek,C.L., Jager,K.J., Dekker,F.W., Zoccali,C., van Diepen,M. (2021). External validation of prognostic models: What, why, how, when and where?Clinical Kidney Journal, 14(1), 49–58.
41.
Robinson,S. (1997). Simulation model verification and validation: Increasing the users’ confidence. In: Proceedings of the 29th Winter Conference on Simulation, 53-59.
42.
Robinson,S. (2003). Modelling human decision-making. In: Proceedings of the 17th European Simulation Multiconference, 448-455.
43.
Rodríguez,N.D., Cuéllar,M.P., Lilius,J., Calvo-Flores,M.D. (2014). A survey on ontologies for human behaviorrecognition. ACM Computing Surveys, 46(4), 1–33.
44.
Rosenfeld,A., Kraus,S. (2018). Predicting human decision-making: From prediction to action. In: Synthesis Lectures on Artificial Intelligence and Machine Learning, Morgan & Claypool, 1-150.
45.
Rothwell,P.M. (2005). External validity of randomised controlled trials: “To whom do the results of this trial apply?”. The Lancet, 365(9453), 82–93.
46.
Ryan,M.J., Wheatcraft,L.S. (2017). On the use of the terms verification and validation. In: Proceedings of the INCOSE International Symposium, 27(1), 1277–1290.
47.
Shahar,N., Hauser,T.U., Moutoussis,M., Moran,R., Keramati,M., NSPN ConsortiumDolan,R.J. (2019). Improving the reliability of model-based decision-making estimates in the two-stage decision task with reaction-times and drift-diffusion modeling. PLOS Computational Biology, 15(2), e1006803.
48.
Silveira,F.R., Campos,G.A., Cortes,M.I. (2013). Rational agents for the test of rational agents. IEEE Latin America Transactions, 11(1), 651–657.
49.
Steyerberg,E.W., Bleeker,S.E., Moll,H.A., Grobbee,D.E., Moons,K.G. (2003). Internal and external validation of predictive models: A simulation study of bias and precision in small samples. Journal of Clinical Epidemiology, 56(5), 441–447.
50.
Steyerberg,E.W., Harrell,F.E., Borsboom,G.J., Eijkemans,M.J., Vergouwe,Y., Habbema,J.D. (2001). Internal validation of predictive models: Efficiency of some procedures for logistic regression analysis. Journal of Clinical Epidemiology, 54(8), 774–781.
51.
Tepjit,S. (2022). Designing an active recommender framework to support the development of reasoning mechanisms for smart cyber-physical systems. Ph.D. Dissertation, Delft University of Technology, Delft, the Netherlands, 1–313.
52.
Tepjit,S., Horváth,I., Rusák,Z. (2019). The state offramework development for implementing reasoning mechanisms in smart cyber-physical systems: A literature review. Journal ofComputational Design and Engineering, 6(4), 527–541.
53.
Thacker,B. H., Doebling,S. W., Hemez,F. M., Anderson,M. C., Pepin,J. E., Rodriguez,E. A. (2004). Concepts of model verification and validation. Technical Report, LA-14167 TRN: US0500431, Los Alamos National Lab., Los Alamos, NM, 1-36.
54.
Thiriot,S., Kant,J.D. (2006). A multi-agent cognitive framework to model human decision making under bounded rationality. In: Proceedings of the International Conference on Behavioural Economics and Economic Psychology, Paris, 1-12.
55.
Tversky,A., Kahneman,D. (2015). Causal schemas in judgments under uncertainty. Progress in Social Psychology, 1, 49–72.
56.
van Duinen,R., Filatova,T., Jager,W., van der Veen,A. (2016). Going beyond perfect rationality: drought risk, economic choices and the influence of social networks. The Annals of Regional Science, 57(2), 335–369.
57.
Wainwright,J., Mulligan,M. (2004). Modelling human decision-making. In: Environmental Modelling: Finding Simplicity in Complexity, John Wiley &Sons, Ltd., 225–244.
58.
Wu,W., Huang,J., Zhang,Z. (2020). Human decision-making behavior modeling for human multi-robot interaction system. IFAC PapersOnLine, it 53(2),
10202–10207.
59.
Yang,Y., Jiang,Y., Zuo,Z., Wang,Y., Sun,H., Lu,H., XuB. (2019). Automatic self-validation for code coverage profilers. In: Proceedings of the 34th International Conference on Automated Software Engineering, IEEE, 79-90.
60.
Yin,C., McKay,A. (2018). Model verification and validation strategies and methods: An application case study. In: Proceedings of the 8th International Symposium on Computational Intelligence and Industrial Applications, 02-06 November, 2018, Tengzhou, China, 1-6.
61.
Yu,C.W., Choi,J.Y. (2005). Behavior decision model based on emotion and dynamic personality. In: Proceedings of the International Conference on Control, Automation and Systems, 2-5 June 2005, Gyeong Gi, Korea, 101-106.
62.
Zha,X.F., Lim,S.Y., Lu,W.F. (2003). A knowledge intensive multi-agent system for cooperative/collaborative assembly modeling and process planning. Journal of Integrated Design and Process Science, 7(1), 99–122.
63.
Zhang,Z., Russwinkel,N., Prezenski,S. (2018). Modeling individual strategies in dynamic decision-making with ACT-R: A task toward decision-making assistance in HCI. Procedia Computer Science, 145, 668–674.
