There is nothing medically magical about machine learning

Coates and de Koning ¹ warn us wisely. An appropriate, well-understood, scientific evidence base is always needed before attempting automation. Racial, gender and socioeconomic bias may creep in, even in critical care training data. Informatic oversight via comprehensive clinical reviews and meta-analyses is essential.

Without a ‘human in the loop’, medicine is not healthcare. Machine learning (ML) is just a complicated statistico-mathematical modelling, and as Box ² says ‘All models are wrong but some are useful’. ML or artificial intelligence (AI) still requires the validating and qualifying steps described ³ as the TRL5-8 stages of development. Not just clinical validity in critical care, but utility, contraindications, cost effectivity, impact on health budgets, operationalisation and continuing quality audit and reviews need examining. Moreover, the right data to probe this critically may not currently be in the digital electronic health records; therefore, specially designed trials may be needed.

Relying solely upon ‘Computer says … yes/no’ is not an acceptable way to deal with human beings, even under the urgent demands of rapid responses. Blind acceptance of AI in clinical decision-making runs the risk of promulgating Automated Idiocy. Informed informatic and trusted clinical judgement must be embedded to challenge, put into context or indeed correct automated decisions.

Patients are not widgets and ML/AI is not exceptional.