This paper presents the Overall Prompting Effectiveness (OPE) framework, designed to enhance interactions with systems employing generative AI(genAI) through prompt engineering. Leveraging principles from Total Productive Maintenance (TPM): Overall Equipment Effectiveness (OEE), the OPE framework provides a systematic tool for iterative refinement of prompts, crucial for optimising human-AI interactions in cyber-physical-social-human systems. Evaluating prompting results in three actionable categories: Availability, Performance, and Quality, OPE aims to generate more accurate, relevant, and effective AI responses. Preliminary applications and workshops with prompt engineers underscore the potential of OPE to significantly improve the efficiency and effectiveness of AI engagements. The integration of OPE within Large Language Models (LLMs) or other genAI systems promises to expand automation capabilities, enhancing the intellectualisation and personalisation of CPS. This framework fosters continuous improvement in prompt quality and contributes to the synthetic knowledge management and smart reasoning mechanisms essential for advanced CPS. Our findings suggest that OPE could play a significant role in cognitive engineering, offering a novel methodology for enhancing decision-making and problem-solving in intelligent systems. This aligns with the current research directions in CPS, emphasising the need for tools supporting technology's socialisation and personalisation in complex integrated environments.
Abdoune, F., Nouiri, M., Castagna, P., Cardin, O. (2021). Toward digital twin for cyber physical production systems maintenance: Observation framework based on artificial intelligence techniques. In: T. Borangiu, D. Trentesaux, P. Leitão, O. Cardin, & S. Lamouri (Eds.), Service oriented, holonic and multi-agent manufacturing systems for industry of the future. SOHOMA 2020. Studies in computational intelligence (Vol. 952, pp. 123–134). Springer. https://doi.org/10.1007/978-3-030-69373-2_8
2.
Abou Eddahab-BurkeF-ZHorváthI. (2024). Deriving manageable transdisciplinary research models for complicated problematics associated with next-generation cyber-physical systems: Part 1 - theoretical and methodological foundation. Transdisciplinary Journal of Engineering & Science, 15, 251–273. https://doi.org/10.22545/2024/00254
3.
AljanabiM.YaseenM. G.AliA. H.MohammedM. A. (2023). Prompt engineering: Guiding the way to effective large language models. Iraqi Journal for Computer Science and Mathematics, 151–155. https://doi.org/10.52866/ijcsm.2023.04.04.012
4.
AminM. M.SchullerB. W. (2024, March 20). On Prompt Sensitivity of ChatGPT in Affective Computing. http://arxiv.org/pdf/2403.14006v1.
5.
ArmentanoM. G.AmandiA. (2009). Recognition of user intentions for interface agents with variable order markov models. User Modeling, Adaptation, and Personalization, 173–184. https://doi.org/10.1007/978-3-642-02247-0_18
6.
BanerjeeA.MaityA.KambojP.GuptaS. K. S. (2024, May 19). CPS-LLM: Large Language Model based Safe Usage Plan Generator for Human-in-the-Loop Human-in-the-Plant Cyber-Physical System. http://arxiv.org/pdf/2405.11458.
7.
BaxterG.RooksbyJ.WangY.Khajeh-HosseiniA. (2012). The ironies of automation: still going strong at 30?ECCE ‘12: Proceedings of the 30th European Conference on Cognitive Ergonomics, 65–71. https://doi.org/10.1145/2448136.2448149
8.
DangH.GollerS.LehmannF.BuschekD. (2023). Choice Over Control: How Users Write with Large Language Models using Diegetic and Non-Diegetic Prompting. International Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3544548.3580969
9.
GuJ.ZhenH.ChenS.BeiramiA.HeB.ZhangG.LiaoR.QinY.TrespV.TorrP. (2023, July 24). A Systematic Survey of Prompt Engineering on Vision-Language Foundation Models. http://arxiv.org/pdf/2307.12980.
10.
GundelakhF.StankevichL.KapralovN.EkimovskiiJ. V. (2019). Cyber-Physical System Control Based on Brain-Computer Interface. International Conference on Cyber-Physical Systems.
HettingerA.RothE.BisantzA. (2017). Cognitive engineering and health informatics: Applications and intersections. Journal of Biomedical Informatics(67), 21–33. https://doi.org/10.1016/j.jbi.2017.01.010
LoL. S. (2023). The art and science of prompt engineering: A new literacy in the information age. Internet Reference Services Quarterly, 27(4), 203–210. https://doi.org/10.1080/10875301.2023.2227621
18.
MazzeiA.AnselmaL.SanguinettiM.RappA.ManaD.HossainM. M.PattiV.SimeoniR.LongoL. (2022). Anticipating user intentions in customer care dialogue systems. IEEE Transactions on Human-Machine Systems, 973–983. https://doi.org/10.1109/THMS.2022.3184400
19.
McKeeD.ClementS. J.AlmutairiJ.XuJ. (2017). Massive-scale automation in cyber-physical systems: Vision & challenges. International Symposium on Autonomous Decentralized Systems, 5–11. https://doi.org/10.1109/ISADS.2017.56
NakajimaS. (1990). Introduction to TPM - Total Productive Maintenance: Total productive maintenance. Productivity Pr.
22.
OliveiraL. M.DiasR.RebelloC. M.MartinsM. A.RodriguesA. E.RibeiroA. M.NogueiraI. B. (2021). Artificial intelligence and cyber-physical systems: A review and perspectives for the future in the chemical industry. Applied Informatics, 2(3), 429–443. https://doi.org/10.3390/ai2030027
23.
PintoG.SilvaF. J. G.BaptistaA.FernandesN.CasaisR.CarvalhoC. (2020). TPM implementation and maintenance strategic plan – a case study. Procedia Manufacturing. https://doi.org/10.1016/j.promfg.2020.10.198
24.
PrellB.GüntherN.Reiff-StephanJ. (2021). Cyber-physical production systems in settings with limited infrastructure. TH Wildau Engineering and Natural Sciences Proceedings, 1, https://doi.org/10.52825/thwildauensp.v1i.6
25.
PrellB.Reiff-StephanJ. (2023). A use case for digital tools in crafts: Simulation and virtual reality for carpentries. SNE Simulation Notes Europe, 33(4), 191–198. https://doi.org/10.11128/sne.33.tn.10667
26.
SchenkelbergK.SeidenbergU.AnsariF. (2020). Analyzing the impact of maintenance on profitability using dynamic bayesian networks. Procedia CIRP, 88, 42–47. https://doi.org/10.1016/j.procir.2020.05.008
27.
SchneiderM. F.MillerM. E. (2018). Operationalized Intent for Communication in Human-Agent Teams. Conference on Cognitive and Computational Aspects of Situation Management, 117–123. https://doi.org/10.1109/COGSIMA.2018.8423992
Velásquez-HenaoJ. D.Franco-CardonaC. J.Cadavid-HiguitaL. (2023). Prompt engineering: A methodology for optimizing interactions with AI-language models in the field of engineering. DYNA. https://doi.org/10.15446/dyna.v90n230.111700 . (Ingeniería de instrucciones: una metodología para optimizar interacciones con Modelos de Lenguaje de IA en el campo de ingeniería)
31.
VicenteK. J. (2006). Cognitive engineering: A theoretical framework and three case studies. International Journal of Industrial and Systems Engineering, 168–181. https://doi.org/10.1504/IJISE.2006.009055
WitteveenS.AndrewsM. (2022, November 21). Investigating Prompt Engineering in Diffusion Models. http://arxiv.org/pdf/2211.15462.
34.
WoodsD.RothE. (1988). Cognitive engineering: Human problem solving with tools. Human Factors: The Journal of Human Factors and Ergonomics Society, 30(4), 415–430. https://doi.org/10.1177/001872088803000404
35.
Zamfirescu-PereiraJ.WongR. Y.HartmannB.YangQ. (2023). Why Johnny Can’t Prompt: How Non-AI Experts Try (and Fail) to Design LLM Prompts. International Conference on Human Factors in Computing Systems. https://doi.org/10.1145/3544548.3581388
36.
ZhouY.MuresanuA. I.HanZ.PasterK.PitisS.ChanH.BaJ. (2022, November 3). Large Language Models Are Human-Level Prompt Engineers. http://arxiv.org/pdf/2211.01910.
