Sage Journals: Discover world-class research

Abstract

The automation of data analysis in the form of scientific workflows has become a widely adopted practice in many fields of research. Data-intensive experiments using workflows enabled automation and provenance support, which contribute to alleviating the reproducibility crisis. This paper investigates the existing provenance models as well as scientific workflow applications. Furthermore, here we not only summarize the models at different levels, but also compare the applications, particularly the blockchain applied to the provenance in scientific workflows. After that, a new design of secure provenance system is proposed. Provenance that would be enabled by the emerging technology is also discussed at the end.

Keywords

Provenance scientific workflows provenance model blockchain

Get full access to this article

View all access options for this article.

References

Baker , Reproducibility crisis, Nature 533(26) (2016), 353–366.

M.M.

Bany Taha ,

Chaisiri and

R.K.L.

Ko , Trusted tamper-evident data provenance, in: 2015 IEEE Trustcom/BigDataSE/ISPA, Vol. 1, 2015, pp. 646–653. doi:10.1109/Trustcom.2015.430.

Bates ,

D.J.

Tian ,

K.R.B.

Butler and

Moyer , Trustworthy whole-system provenance for the linux kernel, Usenix Security Symposium (2015).

Bavoil ,

S.P.

Callahan ,

Crossno ,

Freire ,

Scheidegger ,

C.T.

Silva and

H.T.

Vo , Vistrails: Enabling interactive multiple-view visualizations, IEEE Visualization (2005).

Benabdelkader ,

A.H.C.

van Kampen and

S.D.

Olabarriaga , Prov-man: A prov-compliant toolkit for provenance management, PeerJ (2015).

J.C.

Bennett ,

Bhagatwala ,

J.H.

Chen ,

Pinar ,

Salloum and

Seshadhri , Trigger detection for adaptive scientific workflows using percentile sampling, SIAM Journal on Scientific Computing 38(5) (2016), 240–263. doi:10.1137/15M1027942.

Bhagwat ,

Chiticariu ,

W.-C.

Tan and

Vijayvargiya , An annotation management system for relational databases, Very Large Data Bases, 2004.

Bowers ,

T.M.

McPhillips ,

Riddle ,

M.K.

Anand and

Ludäscher , Kepler/ppod: Scientific workflow and provenance support for assembling the tree of life, 2008.

Braun and

Shinnar , A security model for provenance, 2006.

10.

Braun ,

Shinnar and

Seltzer , Securing provenance, Usenix Security Symposium (2008).

11.

Braun ,

Shinnar and

M.I.

Seltzer , Securing provenance, in: 3rd USENIX Workshop on Hot Topics in Security, Proceedings, San Jose, CA, USA, July 29, 2008, 2008.

12.

Buneman ,

Khanna and

Wang-Chiew , Why and where: A characterization of data provenance, in: International Conference on Database Theory, Springer, 2001, pp. 316–330.

13.

A.S.

Butt and

Fitch , Provone+: A provenance model for scientific workflows, in: International Conference on Web Information Systems Engineering, Springer, 2020, pp. 431–444.

14.

Carata ,

Akoush ,

Balakrishnan ,

Bytheway ,

Sohan ,

Seltzer and

Hopper , A primer on provenance, Communications of The ACM 57 (2014), 52–60. doi:10.1145/2596628.

15.

Chebotko ,

Chang ,

Lu ,

Fotouhi and

Yang , Scientific workflow provenance querying with security views, in: 2008 the Ninth International Conference on Web-Age Information Management, 2008, pp. 349–356. doi:10.1109/WAIM.2008.41.

16.

Chebotko ,

Fei ,

Lin ,

Lu and

Fotouhi , Storing and querying scientific workflow provenance metadata using an rdbms, in: Third IEEE International Conference on e-Science and Grid Computing (e-Science 2007), IEEE, 2007, pp. 611–618. doi:10.1109/E-SCIENCE.2007.70.

17.

Chen ,

Liang ,

Li ,

Qin ,

Mu and

Wang , Blockchain based provenance sharing of scientific workflows, International Conference on Big Data (2018).

18.

Chen ,

Liang ,

Li ,

Qin ,

Mu and

Wang , Blockchain based provenance sharing of scientific workflows, in: 2018 IEEE International Conference on Big Data (Big Data), 2018, pp. 3814–3820. doi:10.1109/BigData.2018.8622237.

19.

Chen ,

Hu ,

Zhu ,

Gao and

Li , Research and popularization of national standard for data provenance descriptive model, Standard Science 4 (2019), 108–112, (in chinese).

20.

Cheney ,

Acar and

Ahmed , Provenance traces, 2008, arXiv preprint arXiv:0812.0564.

21.

Chong , Towards semantics for provenance security, in: TAPP’09 First Workshop on Theory and Practice of Provenance, 2009.

22.

Chong , Towards semantics for provenance security, in: TAPP’09 First Workshop on on Theory and Practice of Provenance, 2009.

23.

Clark ,

Ciccarese and

Goble , Micropublications: A semantic model for claims, evidence, arguments and annotations in biomedical communications, Journal of Biomedical Semantics (2013).

24.

Coelho ,

Braga ,

J.M.N.

David ,

M.A.R.

Dantas ,

Ströele and

Campos , Blockchain for reliability in collaborative scientific workflows on cloud platforms, in: International Symposium on Computers and Communications, 2020.

25.

Cohen-Boulakia ,

Biton ,

Cohen and

S.B.

Davidson , Addressing the provenance challenge using zoom, Concurrency and Computation: Practice and Experience (2008).

26.

Costa ,

Silva ,

de Oliveira ,

K.A.C.S.

Ocaña ,

Ogasawara ,

Dias and

Mattoso , Capturing and querying workflow runtime provenance with prov: A practical approach, Edbt Icdt Workshops (2013).

27.

Cuevas-Vicenttin ,

Ludäscher and

Missier , Provone: A Prov Extension Data Model for scientific workflow provenance, 2016, website https://purl.dataone.org/provone-v1-dev.

28.

S.M.S.

da Cruz ,

M.L.M.

Campos and

Mattoso , Towards a taxonomy of provenance in scientific workflow management systems, in: 2009 Congress on Services-I, IEEE, 2009, pp. 259–266.

29.

R.F.

da Silva ,

Filgueira ,

Pietri ,

Jiang ,

Sakellariou and

Deelman , A characterization of workflow management systems for extreme-scale applications, Future Generation Computer Systems 75 (2017), 228–238. doi:10.1016/j.future.2017.02.026.

30.

S.B.

Davidson and

Freire , Provenance and scientific workflows: Challenges and opportunities, International Conference on Management of Data 2008.

31.

S.B.

Davidson ,

Khanna ,

Roy ,

Stoyanovich ,

Tannen and

Chen , On provenance and privacy, International conference on database theory (2011).

32.

Deelman ,

Gannon ,

Shields and

Taylor , Workflows and e-science: An overview of workflow system features and capabilities, Future Generation Computer Systems (2009).

33.

Demichev ,

Dubenskaya ,

Fedotova ,

Kryukov ,

Polyakov and

N.V.

Prikhod’ko , Provenance Metadata Management in Distributed Storages Using the Hyperledger Blockchain Platform, CEUR Workshop Proceedings, 2019.

34.

Demichev ,

Kryukov and

Prikhodko , The approach to managing provenance metadata and data access rights in distributed storage using the hyperledger blockchain platform, in: 2018 Ivannikov Ispras Open Conference (ISPRAS), IEEE, 2018, pp. 131–136. doi:10.1109/ISPRAS.2018.00028.

35.

Fernando ,

Kulshrestha ,

J.D.

Herath ,

Mahadik ,

Ma ,

Bai ,

Yang ,

Yan and

Lu , Sciblock: A blockchain-based tamper-proof non-repudiable storage for scientific workflow provenance, Color Imaging Conference (2019).

36.

Freire and

C.T.

Silva , Making computations and publications reproducible with vistrails, Computing in Science and Engineering (2012).

37.

Gaignard ,

Skaf-Molli and

Bihouée , From scientific workflow patterns to 5-star linked open data, 2016.

38.

Garijo ,

Corcho and

Gil , Detecting common scientific workflow fragments using templates and execution provenance, International Conference on Knowledge Capture (2013).

39.

Garijo and

Gil , A new approach for publishing workflows: Abstractions, standards, and linked data, in: Proceedings of the 6th Workshop on Workflows in Support of Large-Scale Science, 2011, pp. 47–56. doi:10.1145/2110497.2110504.

40.

Gehani and

Tariq , Spade: Support for provenance auditing in distributed environments, in: ACM/IFIP/USENIX International Conference on Distributed Systems Platforms and Open Distributed Processing, Springer, 2012, pp. 101–120.

41.

Gerhards ,

Skorupa ,

Sander ,

Belloum ,

Vasunin and

Benabdelkader , Hist/plier: A two-fold provenance approach for grid-enabled scientific workflows using ws-vlam, in: 2011 IEEE/ACM 12th International Conference on Grid Computing, 2011, pp. 224–225. doi:10.1109/Grid.2011.39.

42.

Gil ,

Ratnakar ,

Kim ,

Moody ,

Deelman ,

P.A.

González-Calero and

Groth , Wings: Intelligent workflow-based design of computational experiments, IEEE Intelligent Systems (2011).

43.

T.J.

Green ,

Karvounarakis and

Tannen , Provenance Semirings. Symposium on Principles of Database Systems, 2007.

44.

Gupta ,

Tanwar ,

Kumar and

Tyagi , Blockchain-based security attack resilience schemes for autonomous vehicles in industry 4.0: A systematic review, Computers & Electrical Engineering 86 (2020), 106717. doi:10.1016/j.compeleceng.2020.106717.

45.

Hao and

Deng , Research on scientific data sharing management integrating data supervision and data traceability, Information Studies: Theory & Application 41(3) (2018), 6, (in chinese).

46.

Hasan ,

Sion and

Winslett , The case of the fake picasso: Preventing history forgery with secure provenance.

47.

Hull ,

Wolstencroft ,

Stevens ,

Goble ,

Pocock ,

Li and

Oinn , Taverna: A tool for building and running workflows of services, Nucleic Acids Research (2006).

48.

T.D.

Huynh and

Moreau , Provstore: A public provenance repository. International Provenance and Annotation Workshop, (2014).

49.

Ikeda and

Widom , Panda: A system for provenance and data, Bulletin of the Technical Committee on Data Engineering (2010).

50.

Jun and

Xin , Design and implementation of a humanities and social sciences data sharing model: A case study of consortium blockchain, Journal of the China Society for Scientific and Technical Information 38(4) (2019), 14, (in chinese).

51.

C.C.

Kannas ,

Kalvari ,

Lambrinidis ,

C.M.

Neophytou ,

C.G.

Savva ,

Kirmitzoglou ,

Antoniou ,

K.G.

Achilleos ,

Scherf ,

C.A.

Pitta et al., Lisis: An online scientific workflow system for virtual screening, Combinatorial Chemistry & High Throughput Screening 18(3) (2015), 281–295. doi:10.2174/1386207318666150305123341.

52.

Kranjc ,

Podpečan and

Lavrač , Clowdflows: A cloud based scientific workflow platform, in: Joint European Conference on Machine Learning and Knowledge Discovery in Databases, Springer, 2012, pp. 816–819. doi:10.1007/978-3-642-33486-3_54.

53.

K.P.

Kumar and

R.C.

Cherukuri , Securing provenance data with secret sharing mechanism: Model perspective, International Conference Data Science (2019).

54.

Li ,

Huang ,

Wei ,

Lv ,

Liu ,

Dong and

Lou , Searchable symmetric encryption with forward search privacy, IEEE Transactions on Dependable and Secure Computing 18(1) (2019), 460–474. doi:10.1109/TDSC.2019.2894411.

55.

Li ,

Ye ,

Li ,

Wang ,

Lou ,

Hou ,

Liu and

Lu , Efficient and secure outsourcing of differentially private data publishing with multiple evaluators, IEEE Transactions on Dependable and Secure Computing (2020).

56.

Liang ,

Shetty ,

D.K.

Tosh ,

C.A.

Kamhoua ,

Kwiat and

Njilla , Provchain: A blockchain-based data provenance architecture in cloud environment with enhanced privacy and availability, IEEE ACM International Symposium Cluster Cloud and Grid Computing (2017).

57.

Linoy ,

Ray and

Stakhanova , Towards eidetic blockchain systems with enhanced provenance, in: 2020 IEEE 36th International Conference on Data Engineering Workshops (ICDEW), 2020, pp. 7–10. doi:10.1109/ICDEW49219.2020.00-14.

58.

Liu ,

Lu and

Che , A survey of modern scientific workflow scheduling algorithms and systems in the era of big data, in: 2020 IEEE International Conference on Services Computing (SCC), IEEE, 2020, pp. 132–141. doi:10.1109/SCC49832.2020.00026.

59.

Liu ,

Pacitti ,

Valduriez and

Mattoso , A survey of data-intensive scientific workflow management, Grid Computing (2015).

60.

P.M.

Luc Moreau : PROV-DM: The PROV Data Model. version http://www.w3.org/TR/prov-dm/ (2013).

61.

Martinho ,

Domingos and

A.R.

Silva , Supporting authentication requirements in workflows, in: ICEIS, Vol. 3, 2006, pp. 181–188.

62.

Mates ,

Santos ,

Freire and

C.T.

Silva , Crowdlabs: Social analysis and visualization for the sciences, Statistical and Scientific Database Management (2011).

63.

Mattoso and

Glavic , Provenance and annotation of data and processes, 2016.

64.

Melnik , 3. Implementation and Applications, 2004.

65.

Missier ,

S.S.

Sahoo ,

Zhao ,

Goble and

A.P.

Sheth , Janus: From workflows to semantic provenance and linked open data, International Provenance and Annotation Workshop (2010).

66.

Moreau ,

Clifford ,

Freire ,

Futrelle ,

Gil ,

Groth ,

Kwasnikowska ,

Miles ,

Missier ,

J.D.

Myers ,

Plale ,

Simmhan ,

E.G.

Stephan and

J.V.

den Bussche , The open provenance model core specification (v1.1), Future Generation Computer Systems (2011).

67.

Moreau ,

Freire ,

Futrelle ,

R.E.

McGrath ,

Myers and

Paulson , The open provenance model: An overview, International Provenance and Annotation Workshop (2008).

68.

Moreau ,

Missier ,

Belhajjame ,

B’Far ,

Cheney ,

Coppens ,

Cresswell ,

Gil ,

Groth ,

Klyne et al., Prov-Dm: The Prov Data Model, W3C, 2013.

69.

K.-K.

Muniswamy-Reddy ,

D.A.

Holland ,

Braun and

M.I.

Seltzer , Provenance-aware storage systems, in: Usenix Annual Technical Conference, General Track, 2006, pp. 43–56.

70.

Neisse ,

Steri and

Nai-Fovino , A blockchain-based approach for data accountability and provenance tracking, Cryptography and Security (2017), arXiv.

71.

Pancerella ,

Hewson ,

Koegler ,

Leahy ,

Lee ,

Rahn ,

Yang ,

J.D.

Myers ,

Didier ,

McCoy et al., Metadata in the collaboratory for multi-scale chemical science, in: International Conference on Dublin Core and Metadata Applications, 2003, pp. 121–129.

72.

Pasquier ,

Han ,

Goldstein ,

Moyer ,

Eyers ,

Seltzer and

Bacon , Practical whole-system provenance capture. Symposium on Cloud Computing (2017).

73.

Pérez ,

Rubio and

Sáenz-Adán , A systematic review of provenance systems, Knowledge and Information Systems 57(3) (2018), 495–543. doi:10.1007/s10115-018-1164-3.

74.

Pradal ,

Artzet ,

Chopard ,

Dupuis ,

Fournier ,

Mielewczik ,

Negre ,

Neveu ,

Parigot ,

Valduriez et al., Infraphenogrid: A scientific workflow infrastructure for plant phenomics on the grid, Future Generation Computer Systems 67 (2017), 341–353. doi:10.1016/j.future.2016.06.002.

75.

Ram ,

Liu et al., A new perspective on semantics of data provenance, SWPM 526 (2009).

76.

Ramachandran and

Kantarcioglu , Using blockchain and smart contracts for secure data provenance management, Cryptography and Security (2017), arXiv.

77.

Ramachandran and

Kantarcioglu , Smartprovenance: A distributed, blockchain based dataprovenance system, Conference on Data and Application Security and Privacy (2018).

78.

D.D.

Roure ,

Goble and

Stevens , The design and realisation of the experimentmy virtual research environment for social sharing of workflows, Future Generation Computer Systems (2009).

79.

Ruan ,

T.T.A.

Dinh ,

Lin ,

Zhang ,

Chen and

B.C.

Ooi , Lineagechain: A fine-grained, secure and efficient data provenance system for blockchains, Very Large Data Bases (2021).

80.

S.S.

Sahoo and

A.P.

Sheth , Provenir ontology: Towards a framework for escience provenance management, 2009.

81.

M.I.

Seltzer ,

K.-K.

Muniswamy-Reddy ,

D.A.

Holland ,

Braun and

Ledlie , Provenance-Aware Storage Systems, 2005.

82.

Q.-F.

Shao ,

C.-Q.

Jin ,

Zhang ,

W.-N.

Qian and

A.-Y.

Zhou , Blockchain: Architecture and research progress, Chinese Journal of Computers 41(5) (2018), 20.

83.

Simmhan ,

Plale and

Gannon , A survey of data provenance techniques, 2005.

84.

Simmhan ,

Plale and

Gannon , Karma2: Provenance management for data-driven workflows, International Journal of Web Services Research (2008).

85.

Simmhan ,

Plale ,

Gannon and

Marru , Performance evaluation of the karma provenance framework for scientific workflows, International Provenance and Annotation Workshop (2006).

86.

Stoffers , Trustworthy provenance recording using a blockchain-like database, 2017.

87.

D.K.

Tosh ,

Shetty ,

Liang ,

Kamhoua and

Njilla , Consensus protocols for blockchain-based data provenance: Challenges and opportunities, in: 2017 IEEE 8th Annual Ubiquitous Computing, Electronics and Mobile Communication Conference (UEMCON), IEEE, 2017, pp. 469–474. doi:10.1109/UEMCON.2017.8249088.

88.

Vicknair ,

Macias ,

Zhao ,

Nan ,

Chen and

Wilkins , A comparison of a graph database and a relational database: A data provenance perspective, in: Proceedings of the 48th Annual Southeast Regional Conference. ACM SE’10, Association for Computing Machinery, New York, NY, USA, 2010. doi:10.1145/1900008.1900067.

89.

W.A.

Warr , Scientific workflow systems: Pipeline pilot and knime, Journal of Computer-Aided Molecular Design 26(7) (2012), 801–804. doi:10.1007/s10822-012-9577-7.

90.

Wilkinson ,

Dumontier ,

I.J.

Aalbersberg ,

Appleton ,

Axton ,

Baak ,

Blomberg ,

J.-W.

Boiten ,

L.O.B.

da Silva Santos ,

P.E.

Bourne ,

Bouwman ,

A.J.

Brookes ,

Clark ,

Crosas ,

Dillo ,

O.G.

Dumon ,

S.C.

Edmunds ,

C.T.

Evelo ,

Finkers ,

Gonzalez-Beltran ,

A.J.G.

Gray ,

Groth ,

Goble ,

J.S.

Grethe ,

Heringa ,

P.A.C.

’t Hoen ,

Hooft ,

Kuhn ,

Kok ,

J.N.

Kok ,

S.J.

Lusher ,

M.E.

Martone ,

Mons ,

A.L.

Packer ,

Persson ,

Rocca-Serra ,

Roos ,

van Schaik ,

S.-A.

Sansone ,

E.A.

Schultes ,

Sengstag ,

Slater ,

Strawn ,

M.A.

Swertz ,

Thompson ,

van der Lei ,

E.M.

van Mulligen ,

Velterop ,

Waagmeester ,

Wittenburg ,

Wolstencroft ,

Zhao and

Mons (eds), The Fair Guiding Principles for Scientific Data Management and Stewardship, Scientific Data, 2016.

91.

Wittek ,

Lawton ,

Dohndorf ,

Weinert and

Ionita , A blockchain-based approach to provenance and reproducibility in research workflows, in: 2021 IEEE International Conference on Blockchain and Cryptocurrency (ICBC), 2021.

92.

Wood , Ethereum: A secure decentralised generalised transaction ledger, 2013.

93.

Yang ,

D.T.

Michaelides ,

C.M.J.

Charlton ,

W.J.

Browne and

Moreau , Deep: A provenance-aware executable document system, International Provenance and Annotation Workshop (2012).

94.

Zafar ,

Khan ,

Suhail ,

Ahmed ,

Hameed ,

H.M.

Khan ,

Jabeen and

Anjum , Trustworthy data: A survey, taxonomy and future trends of secure provenance schemes, Journal of Network and Computer Applications 94 (2017), 50–68. doi:10.1016/j.jnca.2017.06.003.

95.

Zawoad and

Hasan , Secap: Towards securing application provenance in the cloud, in: 2016 IEEE 9th International Conference on Cloud Computing (CLOUD), IEEE, 2016, pp. 900–903. doi:10.1109/CLOUD.2016.0132.

96.

Zhang ,

Chapman and

LeFevre , Do you know where your data’s been? – Tamper-evident database provenance, in: Workshop on Secure Data Management, Springer, 2009, pp. 17–32. doi:10.1007/978-3-642-04219-5_2.

97.

Zhang ,

Kuc and

Lu , Confucius: A tool supporting collaborative scientific workflow composition, IEEE Transactions on Services Computing (2014).

A survey of provenance in scientific workflow

Abstract

Keywords

Get full access to this article

References