Generation of reusable learning objects from digital medical collections: An analysis based on the MASMDOA framework

Abstract

Learning Objects represent a widespread approach to structuring instructional materials in a large variety of educational contexts. The main aim of this work consists of analyzing the process of generating reusable learning objects followed by Clavy, a tool that can be used to retrieve data from multiple medical knowledge sources and reconfigure such sources in diverse multimedia-based structures and organizations. From these organizations, Clavy is able to generate learning objects that can be adapted to various instructional healthcare scenarios with several types of user profiles and distinct learning requirements. Moreover, Clavy provides the capability of exporting these learning objects through standard educational specifications, which improves their reusability features. The analysis proposed is conducted following criteria defined by the MASMDOA framework for comparing and selecting learning object generation methodologies. The analysis insights highlight the importance of having a tool to transfer knowledge from the available digital medical collections to learning objects that can be easily accessed by medical students and healthcare practitioners through the most popular e-learning platforms.

Introduction

Learning objects represent a widespread approach to structuring instructional materials in a large variety of educational contexts, from primary and secondary school (K-12 levels) to graduate studies and staff training. Medical education is not an exception and this kind of learning resources is used when teaching undergraduate medical courses or training healthcare professionals. The current paper addresses an analysis of how Reusable Learning Objects (RLOs) in the context of medical education can be generated from existing digital collections by using a tool called Clavy. ¹ The analysis proposed takes into account the technical features that make learning objects reusable in different educational scenarios and it is performed by examining those technical features involved when accessing, retrieving, and processing online medical resources by means of open computing interfaces.

Given that the context of medical education covers a wide range of topics, we have focused on the radiology area in order to analyze how the related information can be retrieved, organized, and reused for different learning purposes. Radiology teaching files have served such purposes for a very long time. ² Smith and Castillo ³ described the Internet-accessible radiology teaching file, especially addressed to emergency radiologists. Zaidel et al. ⁴ also presented the concept of a web-based radiology teaching file as a mechanism used by students, residents, physicians and researchers to access Web information associated with clinical studies in this area. D’Alessandro ⁵ commented on the role of the RadiologyEducation ⁶ website as a digital library of educational radiology resources, and related initiatives such as Radiopaedia ⁷ or Radrounds ⁸ offer case catalogs showing how patients are imaged and diagnosed. In parallel, the emergence of the PACS (Picture Archiving and Communication System) tools provides a technology to store radiology images, thus enabling convenient access to such images from multiple modalities and devices. ⁹ These PACS items and other radiographic images have been used to generate digital teaching files and imaging repositories ¹⁰ and, along with the DICOM standard, have been used to automatically create teaching file cases. ¹¹ Some of these educational resources are organized into specialized repositories such as a MIRC ¹² (Medical Imaging Resource Center) for storing teaching files, but their access as RLOs remains unresolved. In summary, there is an extraordinary number of educational resources available in the radiology area but the challenge is how to convert them into learning objects that can be reused in different training or learning scenarios.

The analysis of the Clavy tool’s capabilities to generate RLOs from existing medical digital collections is based on the MASMDOA framework ¹³ used for comparing and selecting learning object generation methodologies. Such an analysis framework, along with MedPix (https://medpix.nlm.nih.gov) as an example of an online clinical case database, makes it possible to examine Clavy’s capabilities, such as retrieving heterogeneous multimedia information sources from external databases, organizing and structuring the content elements gathered or supporting the RLO generation process, in general. The remainder of the work is structured as follows. We start by reviewing some related work about the generation of RLOs in several medical education contexts and, more specifically, from the radiology perspective. Then, we provide some insights into Clavy’s mechanisms used to access medical collections, retrieve and organize their digital contents, and export them in standard formats. Next, we address the MASMDOA-based analysis of the Clavy approach to generating RLOs, using MedPix as a significant case study. It is followed by a discussion on said process of analysis and the criteria applied. We finish with some conclusions and lines of further work.

Literature review

Learning objects have been widely used in medical education as they are considered a “grouping of instructional materials structured to meet a specified educational objective”. ¹⁴ This broad definition allows instructors to create a huge variety of materials under the “learning object” umbrella, but their development requires a minimum set of features to formally characterize them in terms of granularity, self-content, reusability, aggregation, or interoperability. ¹⁵ Analyzing such learning object features is a challenging task in e-learning courses, ¹⁶ and especially so within a medical educational context. In this sense, a particular issue is the question about what constitutes an RLO and how to make these objects reusable in such a context. The student perspective on this kind of learning objects was described by Blake, ¹⁷ who commented on some of the barriers to their usage. The design and analysis of RLOs in healthcare meta-analysis learning has also been described. ¹⁸ Windle and Wharrad ¹⁹ introduced some reflections on the characteristics of RLOs in healthcare education; for example, on the use of context-neutral materials as a way to improve reusable features. They also mentioned the need to apply non-restrictive licensing models and that “reusability also requires that a resource is platform-independent”. An interesting aspect included in the Windle and Wharrad work was related to the development processes which were involved in the generation of RLOs, their granularity level (e.g. 15 min of scheduled learning activity) and the role of communities of practice in these generation processes.

Evans ²⁰ commented on how the development and evaluation of RLOs could enhance the learning experience of international healthcare students. This initiative was launched in a specific educational context related to nursing, and referred to two examples of RLOs available in this context. Again, the multiple possibilities when creating this kind of resources make it difficult to have strict rules that guide such a generation process. Instead, each medical discipline seems to adapt this concept of learning object to its own peculiarities (e.g., testing a virtual learning object in the teaching of radiographic cephalometry, ²¹ using 3D anaglyph images made from stereo image pairs in online Anatomy and Physiology courses, ²² developing interactive resources to teach intramuscular medication administration for nursing undergraduates, ²³ developing web-based resources by mixing multimedia elements like RLOs in the hospital care of people with dementia, ²⁴ or developing resources addressed to occupational therapy education). ²⁵ This diversity of areas in which it is possible to generate this kind of learning objects makes the need to deploy computing tools that can support their generation process evident. Moreover, from the previous RLO examples there is also evidence of diverse multimedia materials and formats which can be aggregated during development. This aggregation property is a crucial feature when generating these learning objects since it requires the extraction of such material from multiple medical knowledge sources. In the case of radiology training, there is a large number of information sources, as shown in the previous section. These sources are usually based on a wide range of multimedia formats and different content features. Even simple resources such as images or video recordings can have a complex component, whether coming from the PACS repositories ²⁶ that store them, standard formats such as DICOM to share imaging information objects, ²⁷ virtual reality methods for transforming clinical imaging data into learning objects ²⁸ or “virtual lectures” by means of animations. ²⁹ In order to support the generation of resources for radiology education, some initiatives have been launched, such as the aforementioned use of a DICOM service to create teaching file cases automatically from PACS ¹¹ or the development of a radiography Digital Teaching Library (DTL). ³⁰ However, as Scarsbrook et al. ³¹ commented, some specific characteristics should be assessed when developing a radiology digital teaching file. Another important aspect in the generation of learning objects is the use of standards to improve their interoperability so that they can be implemented across multiple learning platforms. There are several specifications, such as IMS Content Package or SCORM, which are followed to create interactive e-learning courses ³² or used as references in radiology education. ³³ Some of the SCORM-compliant learning initiatives mentioned in the previous reference are RadMoodle or RadSCOPE^® (Radiology Shareable Content for Online Presentation and Education). Moreover, an authoring tool was designed to transform DICOM images into the SCORM format ³⁴ and the ILIAS platform was deployed using this format to facilitate the reuse of radiology contents. ³⁵ Concerning the integration of external sources (e.g., radiology images) in educational scenarios, a Web application called USRC (University of Saskatchewan Radiology Courseware) ³⁶ was created to link MIRC teaching cases to Blackboard contents and, more recently, the RadEd ³⁷ tool provides a web-based teaching framework that combines the generation of case-based exercises and the integration of image interaction. This interest in “content creation” and its concomitant challenges is one of the main contributions by RadEd and is highly related to the approach proposed in the current work.

In the end, while multiple tools and standard formats are available during the process of generating RLO examples, there are few methodological proposals dealing with the analysis of this kind of process in the healthcare domain. The work of Wharrad and Windle ³⁸ examined several case studies in the development of pharmacology RLOs or learning objects for lifelong healthcare education, in which a theoretical framework was used to evaluate the process and outputs of RLO creation. This framework also provided a toolkit to perform technical assessment of the new content generated and to measure object attributes such as their media richness or interactivity capabilities. Moreover, a qualitative evaluation of RLOs was performed by a set of students and tutors at the Nursing School of the University of Nottingham. ³⁹ In this context, learning objects promoting the achievement of clinical skills were produced using a validated RLO-development methodology ¹⁹ whose outcomes were peer-reviewed by end users to ensure their quality and relevance. An additional paper proposed the transfer of knowledge in health care through “digitally collecting learning experiences” within the Witra Care project ⁴⁰ although its authors barely mentioned the analysis of the process to generate learning objects and simply tagged such objects in eight general attribute categories. In this sense, the use of the MASMDOA framework provides a powerful analysis tool to enable a systematic review of those criteria, which can be involved in RLO generation.

The Clavy approach in RLO generation

This section describes the Clavy approach to generating RLOs from existing medical collections that are available through the Internet. These collections have a high educational value, but they are usually very heterogeneous and do not provide specific mechanisms to generate RLOs. This fact can be illustrated, for instance, in the plethora of systems that have arisen in the radiology area presented above. Thus, the main rationale of the approach is to address this shortcoming by supporting the generation of such RLOs while taking into account the heterogeneity of available medical knowledge sources, the diversity of multimedia materials, and the contents these sources are composed of, or the variety of standards that can be deployed in the e-learning platforms where these instructional contents are used. The approach introduces a versatile mechanism to retrieve information from external collections, reconfigure it according to diverse structures, and finally, generate this kind of learning objects in several educational specification formats. In this way, Clavy allows users with different profiles to generate RLOs according to their specific needs and instructional requirements; for example, in e-learning-based online courses that combine medical case assets with interactive quizzes or forum activities. For this purpose, Clavy is able to retrieve multiple types of medical knowledge information, such as research articles, clinical reports, or imaging databases, that integrate various multimedia formats from plain or structured text to complex images and video recordings. This broad diversity of medical knowledge presents a daunting task in the generation of learning objects to be deployed in courses addressed to different user profiles (e.g. undergraduate medical students or resident physicians-in-training). The capability of Clavy to reconfigure these diverse knowledge resources into multiple structures and organizations, from small or medium-size course units to institutional training content, facilitates their reusability in multiple medical education contexts. Moreover, the possibility provided by Clavy to export diverse instructional contents in the form of standard specifications such as IMS CP (Content Package) or SCORM improves such reusability.

In order to deal with external sources, which are usually poorly structured, Clavy is able to obtain pieces of content from these sources and arrange them in highly specialized digital collections. For this purpose, the Clavy tool supports a three-step workflow in the creation of such collections from heterogeneous external digital resources:

• The first step enables the importation of resources coming from external knowledge sources. Clavy allows the aggregation of content from multiple collections by using different general-purpose plug-ins to import data from standard platforms and formats such as XML documents, JSON files or relational databases, as well as more complex plug-ins targeted to specific collections that have to be accessed through a specific API (Application Programming Interface). This feature lets healthcare experts retrieve pieces of content that are usually poorly structured in external digital medical collections and arrange them in a versatile and powerful organization.

• The second step consists of the curation of the resources retrieved. When a collection is imported in Clavy, experts can access the collection and edit it in its entirety to enrich it or adapt its structure to specific learning scenarios. Therefore, healthcare experts can curate or reconfigure all the information items imported, thus ensuring a coherent and unified structure and reorganizing the collection to satisfy the needs of final users.

• The third step is concerned with the exportation of the previously curated and/or reconfigured collection. Indeed, when the experts consider that the collection is ready, Clavy provides a second kind of plug-in to export the complete collection, or part of its content elements, to third-party systems, for example packaged as a SCORM object.

The workflow presented is based on several items that are part of a Clavy collection:

• A set of digital resources, which include both local files and external resources represented by URLs (Uniform Resource Locators).

• A set of documents, which represent logical aggregations and organizations of resources in terms of hierarchies of element-value pairs.

• A collection schema, which describes the hierarchical organization of content elements in the collection’s documents.

In this way, instead of adopting a specific metadata standard, Clavy promotes the definition of a suitable schema for each particular collection, following a formalism inspired by generalized markup languages like SGML and XML. ⁴¹ Thus, every document in Clavy shares a common structure described by its collection-specific schema, which can be reconfigured by editing said schema. In this way, it is possible to rename element types, remove useless or non-relevant elements, merge two semantically equivalent element types, and change the hierarchical organization of elements. Figure 1 shows an example screenshot that displays the Clavy schema editor applied to a MedPix sample case. In this example, there are three elements in the collection hierarchy, representing Case information, which contains Patient data, the Findings of the case, or the associated Discussion, a Topics item and a Quiz element. These last two elements are represented in a folder format, hiding their inner structure. In this schema view, some of its elements (displayed in red) are atomic and oriented to delimiting descriptive texts. Others, such as Case or Diagnosis, have an additional hierarchical structure associated which can also be edited in a user-friendly way.

Figure 1.

Schema editor in Clavy.

Clavy can be used to build reconfigurable content repositories from a diversity of information sources. This feature enables the selection of certain knowledge items according to a specific need, for example, an educational purpose in a healthcare context, and this selection can be altered in a dynamic way. Moreover, link elements make it possible to include non-hierarchical relationships in the resulting collection structure, which allows users to enrich the knowledge to be represented with references to internal annotations or external resources. The global process is guided through the following steps in the case of building a content repository with a potential educational purpose, which involves both instructors and programmers: ¹

• First, instructors discover digital resources from different sources with educational value suitable to be transformed into learning objects. Then, they use suitable importation plug-ins to import these resources if available, or otherwise they ask programmers to provide such plug-ins. During this starting step, importation plug-ins collect basic content elements and also aggregate these elements by means of hierarchical or non-hierarchical structures. The result in Clavy is an initial metadata schema of the content repository.

• Then, instructors can align the organization of the different content items gathered in the initial metadata schema to get an alternative organization adapted to their own educational requirements. They can also use the document editor to curate some elements by editing their content or adding missed instructional information (e.g. some required learning goals or related assessment strategies). Figure 2 shows an example screenshot displaying the Clavy document editor, which enables information about the Case elements to be changed or inserted; for example, its Diagnosis components, the Findings description or the related Discussion.

• Finally, the repository of content objects can be exported using standard specifications such as IMS CP or SCORM, after deciding the final features of such objects; for example, if they include interactive quizzes or links to image annotations or other multimedia items. For this purpose, instructors can use existing exportation plug-ins, or they can rely on the assistance of programmers to get the exportation functionalities required.

Figure 2.

Document editor in Clavy.

A practical example of the Clavy approach is analyzed in the next section, which shows how this tool can be deployed to generate learning objects that can be reused in different medical contexts.

Analysis of the generation approach

This section addresses the process of analysis regarding how the generation approach proposed can be subjected to a systematic procedure able to retrieve information from heterogeneous medical sources and verify the RLOs obtained. This kind of analysis focuses on those critical aspects that characterize the generation of such learning objects by deploying the Clavy tool. The aspects to be analyzed are based on specific criteria provided in the MASMDOA framework and they are mainly focused on the technological dimension; in particular, the generativity, granularity, interoperability, searchability, and reusability criteria described in this framework. ¹³ An additional component in the methodology of the current research consists of the selection of a significant source of medical contents, which has to comply with two main requirements: (i) allowing free online access by means of automatic information retrieval mechanisms and (ii) offering relevant instructional value which can be useful for healthcare practitioners. There are multiple examples of online educational resources, and Prisciandaro ⁴² provided an exhaustive list of sites addressed to medical physicians and classified into different medical fields. We reviewed some of these sites and selected the MedPix database as a suitable example of a digital medical collection, which included over 12,000 patient case scenarios, 9000 topics, and 59,000 images. Such a medical collection has been used in digital radiology development ³¹ and radiology atlases. ⁴³ After finding a REST-API mechanism to access MedPix contents, we asked their administrators for permission and obtained a subset of cases to show how they could be converted to RLO samples. The generation of these samples was the target of the analysis process whose MASMDOA criteria are described in the following paragraphs.

Generativity

According to the MASMDOA framework, generativity is the criterion that refers to the way in which the structure of the learning object is determined and in which simpler learning objects are assembled to produce more complex ones. The medical domain is characterized by a knowledge corpus composed of a large number of information sources whose structure is highly complex and heterogeneous, such as research articles, clinical reports, or imaging databases. Clavy faces the challenge of generating learning objects from all these items by means of specialized plug-ins able to import information coming from multiple sources with different structure organizations. In the case of MedPix, an importation plug-in was implemented to connect with its API and retrieve a subset of clinical cases (around 6700), 4000 topics, and 1700 questions. Figure 3 shows an excerpt from the initial schema for this collection, which mirrors the basic MedPix case structure in Clavy terms. This initial schema includes data related to the patient such as sex or age, along with more specific information such as the Findings, Diagnosis or Discussion of the case.

Figure 3.

Initial schema for the MedPix collection (excerpt).

Once the content is stored as a Clavy collection, it can be curated by means of the schema and document editors. In particular, Clavy enables the integration of different types of contents that could be part of the learning objects produced. These contents can be external, with references to URL resources, or come from the same Clavy structure model in the form of annotations or links to internal resources (e.g. a list of x-ray images). For instance, Figure 4 shows a screenshot with an example of annotation incorporated in the Clavy document structure, which displays a comment over an injury detected in one of the MedPix case images. This aggregation property is very important when adding multimedia items that enrich the information model of the learning object to be generated.

Figure 4.

Sample of annotation incorporated in a Clavy document.

Granularity

The granularity criterion addresses the definition of larger or smaller learning objects that can be used to support a fine-grain instructional process (for instance, oriented toward undergraduate students in first-year courses who learn basic medical concepts) versus a more detailed information model addressed to representing complex clinical cases to be studied by resident physicians. Clavy deals with granularity by means of schema reconfiguration techniques.

For example, the original MedPix schema outlined in Figure 3 contained 88 elements, many of which were not excessively interesting from an educational point of view (for instance, some technical details of the image list). Figure 5 shows the resulting Clavy schema after such a reconfiguration. The 88 initial elements were reduced to 33 items, many of them grouped in alternative structures. This reconfiguration procedure allows practitioners to choose the most suitable object size according to their educational needs and complement such tailored information with the additional data that makes up the learning object or a specific package thereof.

Figure 5.

Reconfigured schema for MedPix (excerpt).

Interoperability

The next MASMDOA criterion to be analyzed is the interoperability provided by Clavy when generating learning objects under several standard specifications. At this moment, Clavy is able to output collections in the form of IMS Content Packages and SCORM items by using several exportation plug-ins.

Figure 6 shows a screenshot that displays part of a SCORM object obtained from a MedPix case represented through a Clavy document. This object includes Quiz items, which illustrate the possibilities of interaction in the learning objects generated by Clavy.

Figure 6.

SCORM sample about a foot case coming from the MedPix collection.

Searchability

Searchability is mainly based on the use of metadata standards that enable locating or searching for learning objects according to specific features in available repositories or collections. The MASMDOA framework represents this criterion as “locatable” property focused on metadata standards to search for learning objects and locate them. In this sense, Clavy does not currently support the use of these standards so its ability to be searched is limited to the local metadata provided by the same tool.

Reusability

The final criterion to be examined refers to the capability of learning objects which can be used or deployed in learning contexts different from those originally addressed. This capability takes advantage of Clavy’s properties to generate alternative learning object structures, define several granularity levels or aggregate multiple items of multimedia information. For instance, the MedPix example illustrates how Clavy can promote learning object reusability:

• In addition to controlling granularity, schema reconfiguration can be used to reorganize learning object structure. For instance, Figure 5 makes it apparent how information concerning the patient in a MedPix clinical case can be grouped behind a Personal Data structural element, which illustrates how Clavy actually lets instructors change the structure of learning objects and adapt this structure to their particular instructional needs.

• Moreover, the same clinical case can be adapted to show basic case information to be examined or to integrate interactive quizzes to assess user understanding of the case considered, as displayed in Figure 6. That is an example of Clavy’s potential to improve the reusability of those learning objects generated as outcomes, which can be further shared in educational repositories and e-learning platforms.

Discussion

Previous sections have described an approach to generating RLOs based on the Clavy tool and a set of criteria coming from the MASMDOA framework to analyze such a generation approach and its learning object outcomes. The current section addresses a discussion about the issues derived from the suitability of the proposed approach and the adequacy of the analysis criteria to examine the generation of RLOs in this context.

First, the concept of learning object used throughout the article has to be viewed from a wide perspective ¹⁴ that is open to multiple types of instructional materials and addressed to meeting diverse learning goals. Thus, the concept considered is far from being bound to a specific standard with particular metadata requirements, such as those based on the IEEE LOM specification, ³² and the approach proposed fits better with the generation of learning objects whose origin can be found in existing knowledge sources. Generating learning objects is a time-consuming task so we can take advantage of the huge amount of knowledge information in available digital medical collections. Some problems derived from this situation are the high degree of heterogeneity of the aforementioned medical knowledge sources, the diversity of scenarios in which these objects can be implemented or the licensing issues associated with their usage, which were commented on by Windle and Wharrad. ¹⁹ Regarding the heterogeneity of knowledge sources, one of Clavy’s main contributions is the possibility of managing it by means of specialized “importation” plug-ins, in order to allow practitioners to exploit the instructional value of those prestigious digital medical collections that are present in the diverse healthcare disciplines. Dealing with the heterogeneity issue, Clavy has proved to be an efficient “collector” of information that can be poorly structured in origin. This capacity to organize and reconfigure knowledge is another important contribution that enables medical information to be selected or adapted according to user requirements, either for educational or other purposes. Clavy cannot be considered a learning object repository but a tool to organize content items that after a curation process could potentially become learning objects. Thus, in Clavy terms, we are managing content elements that require previous processing in order to be further deployed in a medical learning context. One of the advantages of Clavy is that these content elements can be easily shaped to suit learning requirements in a specific healthcare education scenario. In addition, this shaping or curation process leverages Clavy’s potential to adapt or personalize such elements in a way that improves their reusability. That is a key feature in RLO generation along with the capability to export such content elements to standardized formats that make them interoperable in the several e-learning platforms where healthcare courses can be delivered. The future use of LOD (Linked Open Data) formats will improve the reusability and sharing capability of these medical educational resources. ⁴⁴ Moreover, the possibility of adding or inferring new knowledge from the originally retrieved Clavy collections is another contribution to enhancing the reusability of the learning objects generated. Currently, in Clavy, it is possible to add annotations over the images that are part of a radiology report used for teaching and to include links to external sources such as bibliographic references to electronic medical libraries or online medical databases. In the future, such annotations could be automatically retrieved from DICOM-based imaging systems, or these bibliography references could be extracted through automatic text mining processes in such databases. These features will open new strategies for RLO generation, along with the aggregation of multimedia advances in 3D image processing, virtual reality applications or X-ray simulation tools. ⁴⁵

Of course, there are also limitations to the Clavy approach, which are mainly related to the searchability of its collection contents or the granular definition of the learning objects generated. At this moment, there is little concern about making these collections “searchable” since Clavy is far from being a learning repository and becoming an OER (Open Educational Resources) provider for health sciences, like those mentioned by Minter in the UNMC (University of Nebraska Medical Center) blog. ⁴⁶ Nevertheless, it would be feasible to incorporate some kind of standard learning object model in medical education ⁴⁷ to tag the objects generated, which would improve Clavy’s searchability and also enrich their educational contribution. Moreover, mechanisms for defining the granular level of learning object structures should be included in future versions of Clavy. An additional limitation is the fact that Clavy cannot be considered a learning platform compared to those examples mentioned in section “Literature review” such as USRC ³⁶ or RadEd ³⁷ although it does include features that could help to generate instructional mechanisms such as interactive questionnaires or educational activities linked to multimedia information items. The features provided by Clavy when generating standard specifications are in line with those recommendations for their use in the radiology e-learning context ³³ and in medical education in general. ⁴⁸ In this sense, Clavy’s main purpose in the current work is primarily focused on leveraging this generation process from existing medical digital collections and can be considered its greatest contribution. Finally, from the review of the different quality criteria outlined throughout this “Discussion” section, we can observe the relevance of the MASMDOA framework that provides us with such criteria and a systematic procedure to apply them in order to assess the proposed RLO’s generation approach.

Conclusion

The current work has presented an approach to generating RLOs based on the deployment of a content management tool called Clavy. This approach along with its learning outcomes has been analyzed from a point of view based on a set of criteria adopted from the MASMDOA framework, such as the capacity to import heterogeneous medical knowledge sources, the reconfiguration ability in order to shape such imported resources for different learning structures, the aggregation of various multimedia formats that can be integrated in different levels of learning object granularity or the interoperability of the learning objects generated, which allows them to be used in several e-learning platforms. Clavy has proved to be a suitable tool to produce this kind of learning objects in a user-friendly way, enabling the integration of specialized medical collections in the radiology area. Moreover, this selection of MASMDOA analysis criteria has enabled a systematic procedure to check the generation of RLOs using the Clavy approach, which can be extended in the near future to other healthcare educational domains. Further work will include the possibility of widening the scope of medical collections to be processed or testing the RLOs produced in specific medical learning settings.