In many multiagent systems (MAS), it is desirable that the agents can coordinate with one another on achieving socially optimal outcomes to increase the system level performance, and the traditional way of attaining this goal is to endow the agents with social rationality [in: Proc. of AAAI Fall Symposium on Socially Intelligent Agents, 1997, pp. 61–63] – agents act as system utility maximizers. However, this is difficult to implement when we are facing open MAS domains such as peer-to-peer network and mobile ad-hoc networks, since we do not have control on all agents’ behaviors in such systems and each agent usually behaves individually rationally as an individual utility maximizer only. In this paper, we propose injecting a number of influencer agents [in: Proc. of AAMAS’13, ACM Press, 2013, pp. 447–454, AAMAS (2012)] to manipulate the behaviors of individually rational agents and investigate whether the individually rational agents can eventually be incentivized to coordinate on achieving socially optimal outcomes. We evaluate the effects of influencer agents in two common types of games: prisoner’s dilemma games and anti-coordination games. Simulation results show that a small proportion of influencer agents can significantly increase the average percentage of socially optimal outcomes attained in the system and better performance can be achieved compared with that of previous work.
A.G.Anyouzoa and T.Dhondt, On the stability of a dynamic stochastic capacity pricing scheme for resource allocation in a multi-agent environment, Web Intelligence and Agent Systems3(2) (2005), 85–96.
2.
M.Bowling and M.Veloso, Multiagent learning using a variable learning rate, Artificial Intelligence136 (2002), 215–250.
3.
I.Chao, O.Ardaiz and R.Sanguesa, Tag mechanisms evaluated for coordination in open multi-agent systems, in: Proc. of ESAW’07, 2008, pp. 254–269.
4.
H.Franks, N.Griffiths and S.S.Anand, Learning influence in complex social networks, in: Proc. of AAMAS’13, ACM Press, 2013, pp. 447–454.
5.
H.Franks, N.Griffiths and A.Jhumka, Manipulating convention emergence using influencer agents, AAMAS (2012).
6.
D.Fudenberg and D.K.Levine, The Theory of Learning in Games, MIT Press, 1998.
7.
C.Guttmann, M.Georgeff and I.Rahwan, Collective iterative allocation: Enabling fast and optimal group decision making the role of group knowledge, optimism, and decision policies in distributed coordination, Web Intelligence and Agent Systems8(1) (2010), 1–35.
8.
D.Hales and B.Edmonds, Evolving social rationality for mas using “tags”, in: Proc. of AAMAS’03, ACM Press, 2003, pp. 497–503.
9.
J.Hao and H.-F.Leung, Achieving socially optimal outcomes in multiagent systems with reinforcement social learning, ACM Transactions on Autonomous and Adaptive Systems (TAAS)8(3) (2013), 15.
10.
J.Y.Hao and H.F.Leung, Learning to achieve social rationality using tag mechanism in repeated interactions, in: Proc. of ICTAI’11, 2011, pp. 148–155.
11.
J.Y.Hao and H.F.Leung, Learning to achieve socially optimal solutions in general-sum game, in: Proc. of PRICAI’12, 2012, pp. 88–99.
12.
L.M.Hogg and N.R.Jennings, Socially rational agents, in: Proc. of AAAI Fall Symposium on Socially Intelligent Agents, 1997, pp. 61–63.
13.
J.Hu and M.Wellman, Multiagent reinforcement learning: Theoretical framework and an algorithm, in: Proc. of ICML’98, 1998, pp. 242–250.
14.
G.Jayaputera, A.Zaslavsky and S.Loke, Design, implementation and run-time evolution of a mission-based multiagent system, Web Intelligence and Agent Systems5(2) (2007), 139–159.
15.
V.Könönen, Asymmetric multiagent reinforcement learning, Web Intelligence and Agent Systems2(2) (2004), 105–121.
16.
V.Könönen, Gradient descent for symmetric and asymmetric multiagent reinforcement learning, Web Intelligence and Agent Systems3(1) (2005), 17–30.
17.
X.Li and L.-K.Soh, Hybrid negotiation for resource coordination in multiagent systems, Web Intelligence and Agent Systems3(4) (2005), 231–259.
18.
M.Littman, Markov games as a framework for multi-agent reinforcement learning, in: Proc. of ICML’94, 1994, pp. 322–328.
19.
M.Matlock and S.Sen, Effective tag mechanisms for evolving coordination, in: Proc. of AAMAS’07, 2007, pp. 1–8.
20.
M.Matlock and S.Sen, Effective tag mechanisms for evolving cooperation, in: Proc. of AAMAS’09, 2009, pp. 489–496.
21.
M.Matlock and S.Sen, The success and failure of tag-mediated evolution of cooperation, in: Proc. of LAMAS, Springer, 2005, pp. 155–164.
22.
H.-J.Mosler, Modelling social behavior with a socio psychological simulation approach, Web Intelligence and Agent Systems2(3) (2004), 185–200.
23.
M.Nowak and K.Sigmund, A strategy of winstay, lose-shit that outperforms tit-for-tat in the prisoner’s dilemma game, Nature364 (1993), 56–58.
24.
D.Qi and R.Sun, A multi-agent system integrating reinforcement learning, bidding and genetic algorithms, Web Intelligence and Agent Systems1(3) (2003), 187–202.
25.
M.Scheutz and P.Schermerhorn, Many is more: The utility of simple reactive agents with predictive mechanisms in multiagent object collection tasks, Web Intelligence and Agent Systems3(2) (2005), 97–116.
26.
S.Sen and S.Airiau, Emergence of norms through social learning, in: Proc. of IJCAI’07, 2007, pp. 1507–1512.
27.
L.-K.Soh and X.Li, Investigating adaptive, confidence-based strategic negotiations in complex multiagent environments, Web Intelligence and Agent Systems6(3) (2008), 313–326.
28.
L.Steels, A self-organizing spatial vocabulary, Artificial Life2(3) (1995), 319–392.
29.
D.Villatoro, S.Sen and J.Sabater-Mir, Topology and memory effect on convention emergence, in: Proc. of WI-IAT’09, 2009, pp. 233–240.
30.
J.Y.Wakano and N.Yamamura, A simple learning strategy that realizes robust cooperation better than pavlov in iterated prisoner’s dilemma, J. Ethology19 (2001), 9–15.
31.
C.J.C.H.Watkins and P.D.Dayan, Q-learning, Machine Learning (1992), 279–292.
