Solution selection plays an important role in crowdsourcing and is an imperative work for requesters. However, to the best of our knowledge, there is few studies focus on the problem of solution selection, especially in crowdsourcing contests for innovative tasks. This paper aims to develop a methodology incorporating quality function deployment (QFD) with 2-tuple linguistic method to assist requesters to select the right solution from a large pool of potential solutions efficiently. The methodology includes three phases. The first phase, i.e. pre-selection, is to screen potential solutions by employing the rule of non-compensatory. The second phase is to construct relationships between requester’s requirements and solution features using quality function deployment (QFD), and further to determine the weights of solution features using 2-tuple linguistic weighted average operator and fuzzy weighted average method. The last phase is to evaluate the performance of potential solutions with respect to solution features, and further estimate their overall performance. Finally, an illustrative application case on the crowdsourcing platform-Taskcn is presented to demonstrate the implementation and effectiveness of the proposed approach.
N.H.Thuan, P.Antunes and D.Johnstone, Factors influencing the decision to crowdsource: A systematic literature review, Information Systems Frontiers18(1) 2016, 47–68.
2.
C.K.M.Lee, C.Y.Chan, S.Ho, K.L.Choy and W.H.Ip, Explore the feasibility of adopting crowdsourcing for innovative problem solving, Industrial Management & Data Systems,115(5) (2015), 803–832.
3.
M.K.Poetz and M.Schreier, The value of crowdsourcing: Can users really compete with professionals in generating new product Ideas?Journal of Product Innovation Management29(2) (2012), 245–256.
4.
T.X.Liu, J.Yang, L.A.Adamic and Y.Chen, Crowdsourcing with all-pay auctions: A field experiment on taskcn, Management Science60(8) (2014), 2020–2037.
5.
D.C.Brabham, Motivations for participation in a crowd-sourcing application to improve public engagement in transit planning, Journal of Applied Communication Research40(3) (2012), 307–328.
6.
D.C.Brabham, Moving the crowd at threadless, Information, Communication & Society13(8) (2010), 1122–1145.
7.
E.Schenk and C.Guittard, Towards a characterization of crowdsourcing practices, Journal of Innovation Economics7(1) (2011), 93–107.
8.
N.Luz, N.Silva and P.Novais, A survey of task-oriented crowdsourcing, Artificial Intelligence Review44(2) (2015), 187–213.
9.
A.Kittur, J.V.Nickerson, M.Bernstein, E.Gerber, A.Shaw, J.Zimmerman, et al., The future of crowd work, Proceedings of the 2013 Conference on Computer Supported Cooperative Work, 2013, pp. 1301–1318.
10.
Y.Gong, Estimating participants for knowledge-intensive tasks in a network of crowdsourcing marketplaces, Information Systems Frontiers19(2) (2017), 301–319.
11.
P.G.Ipeirotis, F.Provost and J.Wang, Quality management on Amazon Mechanical Turk, Proceedings of the ACM SIGKDD Workshop on Human Computation, 2010, pp. 64–67.
12.
J.Le, A.Edmonds, V.Hester and L.Biewald, Ensuring quality in crowdsourced search relevance evaluation, In Proceedings of the ACM SIGIR 2010 Workshop on Crowd-sourcing for Search Evaluation, 2010, pp. 17–20.
13.
D.Oleson, A.Sorokin, G.Laughlin, V.Hester, J.Le and L.Biewald, Programmatic gold: Targeted and scalable quality assurance in crowdsourcing, Proceedings of the 11th AAAI Conference on Human Computation, 2011, pp. 43–48.
14.
A.Burmania, S.Parthasarathy and C.Busso, Increasing the reliability of crowdsourcing evaluations using online quality assessment, IEEE Transactions on Affective Computing7(4) (2016), 374–388.
15.
S.Basu Roy, I.Lykourentzou, S.Thirumuruganathan, S.Amer-Yahia and G.Das, Task assignment optimization in knowledge-intensive crowdsourcing, The VLDB Journal24(4) (2015), 467–491.
16.
E.Herrera-Viedma, Modeling the retrieval process for an information retrieval system using an ordinal fuzzy linguistic approach, Journal of the American Society for Information Science & Technology52(6) (2001), 460–475.
17.
F.Herrera and L.Martinez, A 2-tuple fuzzy linguistic representation model for computing with words, IEEE Transactions on Fuzzy Systems8(6) (2000), 746–752.
18.
E.Herrera-Viedma, F.Herrera, L.Martinez, J.C.Herrera and A.G.L.Pez, Incorporating filtering techniques in afuzzy linguistic multi-agent model for information gathering on the web, Fuzzy Sets and Systems148(1) (2004), 61–83.
19.
M.Al-Mashari, M.Zairi and D.Ginn, Key enablers for the effective implementation of QFD: A critical analysis, Industrial Management & Data Systems105(9) (2005), 1245–1260.
20.
V.Bouchereau and H.Rowlands, Methods and techniques to help quality function deployment (QFD), Benchmarking An International Journal7(1) (2000), 8–20.
21.
E.E.Karsak and M.Dursun, An integrated supplier selection methodology incorporating QFD and DEA with imprecise data, Expert Systems with Applications41(16) (2014), 6995–7004.
22.
E.E.Karsak and M.Dursun, An integrated fuzzy MCDM approach for supplier evaluation and selection, Computers & Industrial Engineering82 (2015), 82–93.
23.
R.Ramanathan and J.Yunfeng, Incorporating cost and environmental factors in quality function deployment using data envelopment analysis, Omega37(3) (2009), 711–723.
24.
C.H.Wang and H.S.Wu, A novel framework to evaluate programmable logic controllers: A fuzzy MCDM perspective, Journal of Intelligent Manufacturing27(2) (2016), 315–324.
25.
L.A.Zadeh, The concept of a linguistic variable and its application to approximate reasoning, Information Sciences8(3) (1975), 199–249.
26.
A.Hadi-Vencheh and A.Mohamadghasemi, A fuzzy AHP-DEA approach for multiple criteria ABC inventory classification, Expert Systems with Applications38(4) (2011), 3346–3352.
27.
C.Porcel, A.G.L.Pez-Herrera and E.Herrera-Viedma, A recommender system for research resources based on fuzzy linguistic modeling, Expert Systems with Applications36(3) (2009), 5173–5183.
28.
E.Herrera-Viedma, G.Pasi, A.G.Lopez-Herrera and C.Porcel, Evaluating the information quality of Web sites: A methodology based on fuzzy computing with words, Journal of the American Society for Information Science & Technology57(4) (2006), 538–549.
29.
H.Zhang, Y.Dong and X.Chen, The 2-Rank Consensus Reaching Model in the Multigranular Linguistic Multiple-Attribute Group Decision-Making, IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2018, pp. 1–15.
30.
S.-P.Wan, 2-Tuple linguistic hybrid arithmetic aggregation operators and application to multi-attribute group decision making, Knowledge-Based Systems45 (2013), 31–40.
31.
L.Zhang, Y.Wang and X.Zhao, A new emergency decision support methodology based on multi-source knowledge in 2-tuple linguistic model, Knowledge-Based Systems144 (2018), 77–87.
32.
B.Satzger, H.Psaier, D.Schall and S.Dustdar, Auction-based crowdsourcing supporting skill management, Information Systems38(4) (2013), 547–560.
33.
C.Terwiesch and Y.Xu, Innovation contests, open innovation, and multiagent problem solving, Management Science54(9) (2008), 1529–1543.
34.
P.Cheng, X.Lian, L.Chen, J.Han and J.Zhao, Task assignment on multi-skill oriented spatial crowdsourcing, IEEE Transactions on Knowledge & Data Engineering28(8) (2016), 2201–2215.
35.
Y.Chen, R.Y.K.Fung and J.Tang, Rating technical attributes in fuzzy QFD by integrating fuzzy weighted average method and fuzzy expected value operator, European Journal of Operational Research174(3) (2006), 1553–1566.
36.
F.R.Lima, L.Osiro and L.C.R.Carpinetti, A fuzzy inference and categorization approach for supplier selection using compensatory and non-compensatory decision rules, Applied Soft Computing13(10) (2013), 4133–4147.
37.
J.Rezaei, T.Nispeling, J.Sarkis and L.Tavasszy, A supplier selection life cycle approach integrating traditional and environmental criteria using the best worst method, Journal of Cleaner Production135 (2016), 577–588.
38.
Y.Dong, Y.Wu, H.Zhang and G.Zhang, Multi-granular unbalanced linguistic distribution assessments with interval symbolic proportions, Knowledge-Based Systems82 (2015), 139–151.
39.
Y.Chen, R.Y.K.Fung and J.Tang, Fuzzy expected value modelling approach for determining target values of engineering characteristics in QFD, International Journal of Production Research43(17) (2005), 3583–3604.
40.
A.Kittur, B.Smus and R.Kraut, CrowdForge: Crowdsourc-ing complex work, CHI '11 Extended Abstracts on Human Factors in Computing Systems, 2011, pp. 1801–1806.
41.
A.Kulkarni, M.Can and B.Hartmann, Collaboratively crowdsourcing workflows with turkomatic, Proceedings of the ACM 2012 conference on Computer Supported Cooperative Work, 2012, pp. 1003–1012.
42.
R.Chadawada, A.Sarfaraz, K.Jenab and H.Pourmo-hammadi, Integration of AHP-QFD for selecting facility location, Benchmarking An International Journal22(3) (2015), 411–425.
43.
A.Bhattacharya, J.Geraghty and P.Young, Supplier selection paradigm: An integrated hierarchical QFD methodology under multiple-criteria environment, Applied Soft Computing10(4) (2010), 1013–1027.
44.
Y.Zare Mehrjerdi, M.S.Owlia and A.Tanha, Evaluation and ranking the relative importance of design requirements by combining QFD and DEA techniques (case study: Tile industry of Iran), International Journal of Industiral Engineering & Producion Management23(2) (2012), 175–186.
45.
O.Kilincci and S.A.Onal, Fuzzy AHP approach for supplier selection in a washing machine company, Expert Systems with Applications38(8) (2011), 9656–9664.
46.
Y.Ju, A.Wang and X.Liu, Evaluating emergency response capacity by fuzzy AHP and 2-tuple fuzzy linguistic approach, Expert Systems with Applications39(8) (2012), 6972–6981.
47.
L.F.D.O.M.Santos, L.Osiro and R.H.P.Lima, A model based on 2-tuple fuzzy linguistic representation and Analytic Hierarchy Process for supplier segmentation using qualitative and quantitative criteria, Expert Systems with Applications79 (2017), 53–64.
48.
W.-C.Ko, Construction of house of quality for new product planning: A 2-tuple fuzzy linguistic approach, Computers in Industry73 (2015), 117–127.
49.
J.Dai and J.Blackhurst, A four-phase AHP-QFD approach for supplier assessment: A sustainability perspective, International Journal of Production Research50(19) (2012), 5474–5490.
50.
H.Akbas and B.Bilgen, An integrated fuzzy QFD and TOP-SIS methodology for choosing the ideal gas fuel at WWTPs, Energy125 (2017), 484–497.
51.
A.H.Kumaraswamy, A.Bhattacharya, V.Kumar and M.Brady, An Integrated QFD-TOPSIS Methodology for Supplier Selection in SMEs, 2011 Third International Conference on Computational Intelligence, Modelling & Simulation, 2011, pp. 271–276.
52.
E.Bottani, P.Centobelli, T.Murino and E.Shekarian, A QFD-ANP method for supplier selection with benefits, opportunities, costs and risks considerations, International Journal of Information Technology & Decision Making17(3) (2018), 911–939.
53.
N.A.Tourzani, A.Keramati and A.Apornak, Supplier selection model using QFD-ANP methodology under fuzzy multi-criteria environment, International Journal of Productivity & Quality Management24(1) (2018), 59–83.
54.
X.Zhang, User selection for collaboration in product development based on QFD and DEA approach, Journal of Intelligent Manufacturing (2017). doi: 10.1007/s10845-017-1386-3
55.
H.Zhang, I.Palomares, Y.Dong and W.Wang, Managing non-cooperative behaviors in consensus-based multiple attribute group decision making: An approach based on social network analysis, Knowledge-Based Systems (2018). doi: 10.1016/j.knosys.2018.06.008
56.
Y.Dong, S.Zhao, H.Zhang, F.Chiclana and E.Herrera-Viedma, A self-management mechanism for non-cooperative behaviors in large-scale group consensus reaching processes, IEEE Transactions on Fuzzy Systems (2018), 1–1. doi: 10.1109/tfuzz.2018.2818078
