This paper presents a framework for integrating intrinsic motivation with particle swarm optimisation. Intrinsically motivated particle swarm optimisation can be used for adaptive task allocation when the nature of the target task is not well understood in advance, or can change over time. We first present a general framework in which a computational model of motivation generates a dynamic fitness function to focus the attention of the particle swarm. We then discuss two approaches to modelling motivation in this framework: a computational model of curiosity using an unsupervised neural network and a model of novelty based on background subtraction. We introduce metrics for evaluating intrinsically motivated particle swarm optimisation and test our algorithm as an approach to task allocation in a workplace hazard mitigation scenario. We found that both proposed motivation techniques work well for generating a fitness function that can locate hazards, without requiring a precise definition of a hazard. We found that particle swarm optimisation can converge on optima in our generated fitness landscape in some, but not all, of our simulations.
BahnS. (2013). Workplace hazard identification and management: The case of an underground mining operation. Safety Science, 57, 129–137.
2.
BaldassarreG.MirolliM. (2013a). Intrinsically motivated learning systems: An overview. In Intrinsically motivated learning in natural and artificial systems (pp. 1–14).
3.
BaldassarreG.MirolliM. (Eds.). (2013b). Novelty-based intrinsic motivation mechanisms. In Intrinsically motivated learning in natural and artificial systems (pp. 183–254). Berlin, Heidelberg: Springer.
4.
BaranesA.OudeyerP.-Y. (2012). Active learning of inverse models with intrinsically motivated goal exploration in robots. Robotics and Autonomous Systems, 61, 49–73.
5.
BlackwellT. M. (2005). Particle swarms and population diversity. Soft Computing, 9(11), 793–802.
6.
BloisiD. (2014). Background modeling and foreground detection for maritime video surveillance. Boca Raton, FL: Chapman and Hall/CRC.
7.
BrankeJ. (1999). Memory enhanced evolutionary algorithms for changing optimization problems. Proceedings of the 1999 Congress on Evolutionary Computation, CEC 1999, 3 (721), 1875–1882.
CouceiroM. S.RochaR. P. (2011). A novel multi-robot exploration approach based on particle swarm optimization algorithms. In The 2011 IEEE international symposium on safety, security and rescue robotics, Kyoto, November 1–5 (pp. 327–332). IEEE.
10.
EberhartR.KennedyJ. (1995). A new optimizer using particle swarm theory. MHS’95. Proceedings of the Sixth International Symposium on Micro Machine and Human Science, 1, 39–43.
11.
EberhartR.ShiY. (2000). Comparing inertia weights and constriction factors in particle swarm optimization. In Proceedings of the 2000 congress on evolutionary computation, La Jolla, CA, July 16–19 (Vol. 1, pp. 84–88). IEEE.
12.
FengY.WuZ.-F.WuK.-G.XiongZ.-Y.ZhouY. (2005). An unsupervised anomaly intrusion detection algorithm based on swarm intelligence. In Machine learning and cybernetics, 2005. proceedings of 2005 international conference on, Guangzhou, China, August 18–21 (Vol. 7, pp. 3965–3969). IEEE.
13.
Fernandez-MarquezJ.ArcosJ. (2010). Adapting particle swarm optimization in dynamic and noisy environments. In 2010 IEEE congress on evolutionary computation (CEC), Barcelona, July 18–23 (pp. 1–8). IEEE.
14.
GrubbsF. (1969). Procedures for detecting outlying observations in samples. Technometrics, 11, 1–21.
15.
HardhienataM. K. D.MerrickK. E.UgrinovskiiV. (2014, December). Effective motive profiles and swarm compositions for motivated particle swarm optimisation applied to task allocation. In 2014 IEEE symposium on computational intelligence for human-like intelligence (CIHLI), Orlando, Florida, December 9–12 (pp. 1–8). IEEE.
16.
HassanR.CohanimB.De WeckO.VenterG. (2005). A comparison of particle swarm optimization and the genetic algorithm. In 46th AIAA/ASME/ASCE/AHS/ASC structures, structural dynamics, and materials conference, Austin, Texas, April 18–21 (pp. 1–13). AIAA.
17.
HeckhausenJ.HeckhausenH. (2010). Motivation and action. New York, USA: Cambridge University Press.
18.
HodgeV.AustinJ. (2004). A survey of outlier detection methodologies. Artificial Intelligence Review, 22, 85–126.
19.
KarimiJ.NobahariH.PourtakdoustS. H. (2012). A new hybrid approach for dynamic continuous optimization problems. Applied Soft Computing Journal, 12 (3), 1158–1167.
20.
KennedyJ.EberhartR. (1995). Particle swarm optimization. Proceedings of ICNN’95 - International Conference on Neural Networks, 4, 1942–1948.
21.
KlyneA.MerrickK. (2015). Task allocation using particle swarm optimisation and anomaly detection to generate a dynamic fitness function. In 28th Australasian joint conference on artificial intelligence 2015, Australia, Springer, November 30–December 4 (pp. 317–329). Canberra.
KohonenT. (1990). The self-organizing map. Proceedings of the IEEE, 78 (9), 1464–1480.
24.
LehmanJ.StanleyK. (2011). Abandoning objectives: Evolution through the search for novelty alone. Evolutionary Computation, 19(2), 189–223.
25.
LiX. L. X.Hong-XuM. H.-X. M. (2008, June). Particle swarm optimization based multi-robot task allocation using wireless sensor network. 2008 international conference on information and automation, Changsha, June 20–23 (pp. 1300–1303). IEEE.
26.
LiZ.LiY.XuL. (2011). Anomaly intrusion detection method based on K-means clustering algorithm with particle swarm optimization. 2011 International Conference of Information Technology, Computer Engineering and Management Sciences, 2, 157–161.
27.
LinK.ChenS.-C.ChenC.-S.LinD.-T.HungY.-P. (2014). Left-luggage detection from finite-state-machine analysis in static-camera videos. 2014 22nd international conference on pattern recognition, Stockholm, August 24–28 (pp. 4600–4605). IEEE.
28.
MarkouM.SinghS. (2003a). Novelty detection: A review - Part 1: Statistical approaches. Signal Processing, 83 (12), 2481–2497.
29.
MarkouM.SinghS. (2003b). Novelty detection: A review - Part 2: Neural network based approaches. Signal Processing, 83 (12), 2499–2521.
30.
MarslandS. (2011). Machine learning: An algorithmic perspective (1st ed.). New York, NY: CRC Press.
31.
MarslandS.NehmzowU.ShapiroJ. (2004). Online novelty detection for autonomous mobile robots. Journal of Robotics and Autonomous Systems, 51, 191–206.
32.
MerrickK.MaherM. L. (2009). Motivated learning from interesting events: Adaptive, multitask learning agents for complex environments. Adaptive Behavior, 17, 7–27.
33.
MerrickK.MaherM. L.SaundersR. (2008). Achieving adaptable behaviour in intelligent rooms using curious supervised learning agents. In Caadria 2008 beyond computer aided design, Chiang Mai, Thailand (pp. 185–192). CAADRIA.
34.
MiljkovicD. (2010). Review of novelty detection methods. In Mipro, 2010 proceedings of the 33rd international convention, Opatija, Croatia, May 24–28 (pp. 593–598). IEEE.
35.
MorrisonR. W.De JongK. A. (2002). Measurement of population diversity. Artificial Evolution, 1 (2), 31–41.
36.
NguyenT. T.YangS.BrankeJ. (2012). Evolutionary dynamic optimization: A survey of the state of the art. Swarm and Evolutionary Computation, 6, 1–24.
SaundersR. (2002). Curious design agents and artificial creativity: A synthetic approach to the study of creative behaviour. Unpublished doctoral dissertation, University of Sydney, Australia.
39.
SinghS.BartoA.ChentanezN. (2004). Intrinsically motivated reinforcement learning. In Advances in neural information processing systems (NIPS), Vancouver, British Columbia, Canada, December 13–18 (pp. 1281–1288). NIPS 17.
40.
WangB.MerrickK.AbbassH. (2015). Autonomous hypothesis generation as an environment learning mechanism for agent design. Australian conference on artificial life and computational intelligence (pp. 210–225).
41.
WengJ.McClellandJ.PentlandA.SpornsO.StockmanI.SurM.ThelenE. (2001). Autonomous mental development by robots and animals. Science, 5504(291), 599–600.
42.
YangS.LiC. (2010). A clustering particle swarm optimizer for locating and tracking multiple optima in dynamic environments. IEEE Transactions on Evolutionary Computation, 14(6), 959–974.
