Sage Journals: Discover world-class research

Abstract

Digital Learning Resources (DLRs) have the function of facilitating learning; however, the evaluation regarding their functionality is limited since there are few tools for this purpose. Of these, the instrument that stands out is “Perception of the use of Digital Learning Resources in Statistics” (IPDLR-S), which allows for evaluating the perception of undergraduate university students about the functionality of the LDRs. With the objective of providing a tool with scientific quality, the IPDLR-S instrument underwent to an analysis of construct validity and reliability. The instrument was applied to 1,148 university students, the construct validity analysis was performed using an exploratory (EFA) and confirmatory (CFA) factor analysis, and the reliability was determined by ordinal Cronbach’s alpha. According to the EFA, it was found that all the items are represented in the factorial model; however, differences were found for the theoretical model since a fourth dimension emerges (Cooperative Learning). On the other hand, using the CFA, the resulting factorial model showed a good fit (GFI = 0.999, RMSEA = 0.064, CFI = 0.999, and TLI = 0.999). Overall reliability, as well as by factor showed an optimal value (ordinal Cronbach’s alpha > .98). The IPDLR-S instrument has adequate psychometric properties.

Keywords

learning objects construct validity statistics cooperative learning

Introduction

Higher education institutions have had to generate changes in teaching-learning processes, not only in methodologies but also by integrating Information and Communication Technologies (ICT) into them (Adedokun-Shittu & Shittu, 2015). The integration of ICT into classrooms not only implies changes both in the curriculum and in their facilities, but also an invitation to pedagogical innovation (Sánchez, 2019). In the educational context, ICTs are important for students and should be used effectively in learning, as well as, for the teachers who must integrate them into the classroom (Deshpande & Shesh, 2021); this is why the integration of ICT in the university is favorable for learning and teaching (Adedokun-Shittu & Shittu, 2015).

Given the integration of ICT, the use of Learning Objects (LO) stands out, which have been considered functional to improve students learning (Kay & Knaack, 2009). In this sense, LOs are referred to as a digital representation that allows uses in different educational contexts, use different modalities of media (and often interactivity) to represent data, information, concepts, and ideas and also are designed to allow educational reuse (Churchill, 2007). From this perspective, Churchill (2007) refers that, for teachers, an LO allows its use in a variety of foreseen and unforeseen circumstances; in this sense, LO can be used as: 1. a component in the direct instruction for online delivery, 2. a mediating instrument in a problem-solving activity, 3. the object of an investigation, 4. digital resources so that students can use them in their studies, assignments and independent projects, and 5. models and means that the LO designers can use as a base for the design of other’s LO. The LO can be delivered through various digital devices; this is how those who design an LO may perceive it as applicable to a specific educational use.

Within the LO, there is the Digital Learning Resources (DLR), which consist of practice LO and are designed based on the development of practical exercises with feedback, and include graphs and other resources such as tables (Churchill, 2007). In the educational context, there are various types of DLR as those provided by Salajan et al. (2009) designed for the learning of dentistry, Hortense et al. (2018) designed and aimed at patients in cancer treatment, and by Basuhail (2019) designed for teaching programming to computer science students.

It has been indicated that digital resources and specifically LO, can enhance or improve student learning (Kay & Knaack, 2009); therefore, evidence of their effectiveness from the perspective of the student for learning support becomes especially important (Chiu & Churchill, 2016; Garay Ruiz et al., 2017). However, there are few instruments to assess the students’ perceptions of the LO, only including Kay and Knaack’s (2009), which is aimed at high school students, and the one from Monge-Rogel et al. (2022), which is aimed at students of higher education.

In this regard, the contribution of Monge-Rogel et al. (2022) is called “Instrument of Perception of the use of Digital Learning Resources in Statistics” (IPDLR-S), consists of 32 items and 3 dimensions: 1. Functionality and Accessibility of the DLR; 2. Design and Structure of the DLR; 3. Learning statistics. In this order, these dimensions are relevant since, respectively they are intended to obtain information about the ease of access and navigation, interaction in a context applied to the professional field, and the contribution to the learning of statistics based on challenging and motivating activities, oriented to the interaction between peers and teachers, in a constructive alignment plane (Barattucci, 2017).

An aspect to highlight of the IPDLR-S instrument is that after its construction, it was subjected to the analysis of the psychometric property of content validity (Monge-Rogel et al., 2022), through which it was demonstrated that the dimensions and items that make it up are relevant, pertinent and representatives of the construct to be evaluated (Koller et al., 2017; Juárez-Hernández & Tobón, 2018). This is relevant since, as Carvajal et al. (2011) refer to, the evaluation of the psychometric properties of an instrument is an essential criterion to determine the quality of its measurement.

While content validity is a relevant psychometric property, it is emphasized that the main type of validity is the construct (Furr, 2020), which is conceptualized as the degree to which inferences can be made legitimately from operationalizations in their study to the theoretical constructions on which those operationalizations were based. Its importance lies in minimizing the problems that arise from the poor quality of the measurement and in proving that the real internal structure is the one it should possess (Furr, 2020).

Therefore, this study’s objective was to assess the psychometric properties of construct validity and reliability of the IPDLR-S instrument to collect valid and reliable evidence on the perception of the use of DLRs in the learning of statistics.

Methodology

Type of Study

According to the classification of Ato et al. (2013), in this work, an instrumental study was carried out to analyze the construct validity and reliability of the instrument to evaluate the perception of students about learning objects in statistics.

Instrument

The purpose of the IPDLR-S instrument (Monge-Rogel et al., 2022) is to evaluate the perception of undergraduate students about the use of DLR in a statistics subject. The instrument consists of 32 items grouped in three dimensions: Functionality and Accessibility of the DLR (7 items), Design and Structure of the DLR (13 items), and Learning of the Statistics (12 items); and for the answers, it uses the Likert scale of six options (1 = strongly disagree, 2 = disagree, 3 = moderately disagree, 4 = moderately agree, 5 = agree, and 6 = strongly agree).

As it was indicated, after its construction, the instrument was submitted to a trial of 31 experts, through which its validity of content was determined (Monge-Rogel et al., 2022). Once this phase was concluded, a pilot was carried out with 50 undergraduate students of a statistics subject, through which the reliability of the instrument and the degree of affordability were analyzed, resulting in high reliability (ordinal Cronbach’s alpha = .963) and a high degree of acceptance and satisfaction with the instrument (Monge-Rogel et al., 2022).

Selection of a Sample of the Population for the Application of the Instrument

In this study, a sample of 1,148 students of the Veterinary Medicine, Nursing, and Agronomy careers of a Chilean university were considered, who took a subject of statistics and used the DLR. Participants were selected by intentional sampling according to the purpose of the study. The intentional sampling method is considered a non-probabilistic sampling whose main purpose is to produce a sample that can be assumed to be logically representative of the population (Bhardwaj, 2019; Jamalzadeh et al., 2021).

The participants were informed of the availability of the instrument, which was applied in Spanish, and it was placed in the virtual classroom of the course and was available for a month; the student could access the instrument through a link enabled in the same virtual classroom. The sociodemographic data of the participants are shown in Table 1.

Table 1.

Sociodemographic Data of the Participants (n = 1,148).

Variable	Measure
Sex (%)	Women: 75.52
Sex (%)	Men: 24.48
Age (mean ± SD)	24.50 (±6.15) years
Region (%)	Biobío: 14.20
	Metropolitan of Santiago: 67.33
	Valparaiso: 18.47
Career (%)	Agronomy: 2.09
	Nursing: 44.86
	Veterinary Medicine: 53.05

This study had the approval of the Ethics Committee of the institution, complying with all safety and reliability protocols, in addition to obtaining the informed consent of all participants who were included in this study.

Construct Validity and Reliability Analysis

Item Analysis

Initially, the adjustment of the items to the univariate normal distribution was analyzed by calculating the asymmetry and kurtosis, considering that if any item had a value greater than ±2 it was subject to elimination (Bollen & Long, 1993). Likewise, the analysis of multivariate normality was considered using the Mardia coefficient (Mardia, 1970) and under the criteria of Bollen (1989). The item-test correlation was examined to verify whether the items revealed a value less than .2 or a value higher than .9, which led to their elimination (Kline, 1986; Tabachnick & Fidell, 2001). In addition, ordinal Cronbach’s alpha was calculated to evaluate the reliability of the instrument, and an item-by-item contribution analysis was performed on ordinal Cronbach’s alpha; to verify that there were no increases in the value of trustworthiness when not considering one of the items.

Factor Analysis

According to the sample size, a cross-validation process was carried out, for which the sample was divided into two equal parts (n₁ = n₂ = 574), the first of which was analyzed by the Exploratory Factor Analysis (EFA) and the second using the Confirmatory Factor Analysis (CFA; Brown, 2015). The mechanism for the division of the sample was through a randomization process, which is used to avoid any bias or pattern, using random numbers through the statistical R software.

Exploratory Factor Analysis

In order to perform the EFA, first, the relevance of the data was verified through the degree of correlation between items, the value of the determinant, KMO index, and Barlett test (Juárez-Hernández, 2018; Thomson, 2004). When it was confirmed that the items were adjusted to the normal distribution, the EFA was carried out by selecting the Maximum likelihood estimation method (Howard & Jayne, 2015; Thomson, 2004; Yong & Pearce, 2013). However, if the absence of normality was verified, it was chosen to work with the polychoric correlation matrix, and the chosen extraction method was the unweighted least square (ULS; Xia & Yang, 2019). The determination of the number of factors to be retained was based on the technique of maximum consensus among 23 methods used (Gutman-Kaiser rule, sedimentation graph, parallel analysis, optimal coordinates, and variance explained, among others; Lüdecke et al., 2020). Following the analysis of the factorial matrix, if factorial complexity was presented, the rotation of the matrix was carried out using the algorithm of greater convenience (Juárez-Hernández, 2018; Lloret et al., 2017; Thompson, 2004). These analyses were performed with the statistical R software and the Parameters library version 0.14.0.1 (Lüdecke et al., 2020).

Confirmatory Factor Analysis

According to the factorial structure obtained through the EFA, with the second subsample, the CFA was performed using the Robust Unweighted Least Square method. Specifically, the goodness of fit of the model was evaluated using chi-square, ratio, and adjustment index (χ²/gl Goodness of Fit Index (GFI); Mean Square Error of approximation (RMSEA), Root Mean Square Residual (RMSR); Comparative Fit Index (CFI); Index de Tucker-Lewis (TLI)), according to the criteria proposed by Blunch (2012), Schumacker and Lomax (2016), Xia and Yang (2019), and Ráczová et al. (2021). This analysis was performed with the statistical R software and Laavan (Rosseel, 2012) and psych (Revelle, 2017) library.

Divergent and Discriminant Validity

To evaluate whether the behavior of a factor is strongly explained by the items that compose it, convergent validity was explored through the Extracted Mean-Variance (AVE), specifying that an AVE greater than or equal to .5 is considered adequate according to the criterion of Fornell and Larcker (1981). On the other hand, to evaluate the ability to discriminate different constructs, the discriminant validity was analyzed by comparing the correlation squared (r²) with the AVE of each factor; when r² was lesser than the AVE of each factor, the discriminant validity was verified (Fornell & Larcker, 1981).

Instrument Reliability and Composite Reliability

The internal consistency of the instrument was evaluated through the ordinal Cronbach’s alpha reliability, as well as the composite reliability (CR), considering a threshold level greater than .70 as the minimum acceptable value (DeVellis, 2016).

Instrument Satisfaction Analysis

Finally, the degree of understanding and satisfaction with respect to the instructions, items, and descriptors of the instrument was evaluated through a satisfaction questionnaire (Centro Universitario, 2018).

Results

Construct Validity and Reliability Analysis

Although the values of the asymmetry and kurtosis statisticians could suggest that the items are distributed in a normal way (Table 2), the evaluation of multivariate normality was not satisfactory according to the Mardia test (Kurtosis p < .05; Asymmetry p < .05), so robust unweighted least squares were used as the extraction method in the EFA. According to the item-test correlation indicator and the ordinal Cronbach’s alpha, it was not necessary to remove any items (Table 2).

Table 2.

Item Statistics.

Item	M	SD	Coefficient of asymmetry	Kurtosis	Item-test correlation	Ordinal Cronbach’s alpha
1	4.45	1.55	−.84	−0.33	.77	.98
2	4.61	1.40	−.93	0.11	.82	.98
3	4.33	1.41	−.74	−0.12	.80	.98
4	4.54	1.46	−.96	0.11	.83	.98
5	4.63	1.47	−1.03	0.20	.80	.98
6	4.63	1.36	−1.05	0.51	.86	.98
7	4.62	1.43	−1.02	0.26	.87	.98
8	4.66	1.34	−1.04	0.53	.89	.98
9	4.83	1.27	−1.22	1.07	.90	.98
10	4.84	1.27	−1.22	1.05	.90	.98
11	4.81	1.30	−1.19	0.92	.91	.98
12	4.78	1.29	−1.15	0.87	.91	.98
13	4.95	1.25	−1.39	1.60	.87	.98
14	4.85	1.28	−1.29	1.31	.90	.98
15	4.92	1.25	−1.38	1.62	.88	.98
16	4.84	1.26	−1.28	1.34	.90	.98
17	4.82	1.31	−1.26	1.13	.90	.98
18	4.87	1.27	−1.34	1.42	.90	.98
19	4.45	1.44	−.80	−0.19	.70	.98
20	4.71	1.29	−1.09	0.79	.90	.98
21	4.66	1.36	−1.11	0.67	.89	.98
22	4.57	1.40	−.99	0.36	.89	.98
23	3.20	1.92	.12	−1.53	.41	.98
24	4.62	1.42	−1.02	0.32	.85	.98
25	4.59	1.41	−1.02	0.32	.88	.98
26	4.70	1.32	−1.08	0.70	.83	.98
27	4.81	1.27	−1.28	1.34	.88	.98
28	4.46	1.50	−.89	−0.10	.83	.98
29	3.20	1.88	.12	−1.49	.37	.99
30	2.95	1.93	.34	−1.45	.32	.99
31	4.60	1.42	−1.04	0.35	.88	.98
32	4.56	1.46	−1.00	0.20	.88	.98

Factor Analysis

Exploratory Factor Analysis

In specifics, the data were shown to be relevant to be analyzed through exploratory factor analysis (Determinant: 3.258866e-23; KMO = 0.98; χ² = 29,073.39, df = 496; p < .001).

According to the EFA, it was found that all the items presented adequate communalities (Table 3). The matrix revealed the complexity of a series of items, so a rotation was made, clarifying the factor loadings (FL; Table 3). Specifically, it is reported that the resulting factorial model differs from the proposed theoretical model since four factors were denoted, which explained more than 83% of the variance, which is highly representative, and in which all items with significant factorial loads were integrated (FL > 0.50; Table 3). In this sense, the first factor explained more than 35% of the variance and integrated items 8 to 20 (Table 3), corresponding to the theoretical dimension of LDR Design and Structure. Factor two explained 8% of the variance and integrated items 23, 29, and 30 (Table 3), which address the aspects of realization of the LDR by students, either with the help of their peers or the teacher, and emphasized that these items belonged to the theoretical dimension of Learning Statistics. According to the conjunction of these items, the new factor was called “Cooperative Learning.” Factor three explained 18% of the variance and integrated items 1 to 7 (Table 3); this series of items corresponded to the theoretical dimension LDR Functionality and Accessibility. Factor four explained 22% of the variance and integrated items 21, 22, 24, to 28 along with 31 and 32 (Table 3); this series of items corresponded to the theoretical dimension of Learning Statistics.

Table 3.

Communalities and Factor Loadings by Items.

Item	Communality	F1	F2	F3	F4
1	0.71			0.90
2	0.82			0.86
3	0.79			0.93
4	0.87			0.94
5	0.69			0.64
6	0.85			0.74
7	0.87			0.77
8	0.82	0.74
9	0.89	0.82
10	0.86	0.84
11	0.87	0.80
12	0.86	0.74
13	0.88	0.93
14	0.91	0.92
15	0.89	0.91
16	0.89	0.88
17	0.90	0.88
18	0.88	0.86
19	0.62	0.73
20	0.88	0.90
21	0.84				0.85
22	0.87				0.81
23	0.72		0.77
24	0.82				0.98
25	0.87				0.92
26	0.71				0.62
27	0.83				0.67
28	0.76				0.80
29	0.91		0.96
30	0.86		0.87
31	0.82				0.80
32	0.88				0.93

Confirmatory Factor Analysis

The adjustment measures of the CFA showed a good fit for the model (Table 4). More precisely, the ratio χ²/gl, the absolute indices (GFI and RMSEA; Table 4), and the incremental indices (CFI and TLI), and the RMSR showed optimal values (Table 4). The final model is shown in Figure 1.

Table 4.

Adjustment of the Factorial Model.

Indexes	Expected value^a	Value obtained
χ²/gl	2–4	3.37
The goodness of fit index (GFI)	Greater than 0.90	0.999
Root mean square error of approximation (RMSEA)	0.050–0.080	0.064
Root mean square residual (RMSR)	Less than 0.050	0.037
Comparative fit index (CFI)	Greater than 0.95	0.999
Tucker-Lewis index (TLI)	Greater than 0.90	0.999

Qiu et al. (2022).

Figure 1.

Confirmatory factor analysis model of the IPDLR-S instrument.

The resulting factorial model can be seen in Figure 1. As shown, all the items presented high factor loadings (FL > 0.50). Correlations between latent variables are shown with two-way arrows. The IPDLR-S consists of the latent variables: Functionality and Accessibility of the RDR (FA), RDR Design and Structure (DE), Learning Statistics (LS), and Cooperative Learning (CL).

Convergent and discriminant validity

It was found that the four dimensions presented convergent validity since the AVE were greater than .50 $(AV E_{FA} = 0.7999; AV E_{DE} = 0.8375; AV E_{AE} = 0.8488; AV E_{AC} = 0.8099)$ . Likewise, it is stated that the existence of discriminant validity was denoted since the AVE of each dimension is greater than their corresponding correlations with the other dimensions (Table 5).

Table 5.

Discriminant Validity of IPDLR-S With the Second Sample ( $n_{2} = 574$ ).

Dimensions	Functionality and accessibility	Design and structure	Learning statistics	Cooperative analysis
Functionality and accessibility	0.7999	—	—	—
Design and structure	0.751689	0.8375	—	—
Learning statistics	0.682276	0.788544	0.8488	—
Cooperative analysis	0.071289	0.049284	0.1089	0.8099

Note. The values in bold correspond to the AVE, and the non-bold correspond to the correlation between the dimensions (r²).

Instrument Reliability and Composite Reliability

Regarding the reliability of the instrument, an optimal value was obtained (ordinal Cronbach’s alpha: .98). Also, all four factors demonstrate reliability well above the minimum acceptable (CR > .7; Table 6).

Table 6.

Composite Reliability Analysis for IPDLR-S Dimensions With the Second Sample ( $n_{2} = 574$ ).

Dimensions	Number of items	Composite reliability
Functionality and accessibility	7	.9806
Design and structure	13	.9272
Learning statistics	9	.9852
Cooperative analysis	3	.9656

Instrument satisfaction analysis

Finally, the analysis of satisfaction with the instrument (Table 7) revealed a good degree of understanding of the instructions and items by the students.

Table 7.

Frequency Distribution of Satisfaction With the IPDLR-S Instrument (n = 1,148).

Questions	Low degree (%)	Acceptable degree (%)	Well degree (%)	Excellent degree (%)
What was the degree of understanding and acceptance of this instrument?	4.01	16.38	43.90	35.71
What was the degree of understanding of the item questions?	3.22	15.59	45.47	35.72
What was the degree of satisfaction with this instrument?	6.27	16.03	42.25	35.45

Discussion

The COVID-19 pandemic has forced the implementation of new teaching methods strongly supported by new information technologies. Therefore, emergency teaching (Kohnke et al., 2021; Xu et al., 2021) involves the use of LO in online classes.

As indicated, the DLRs (as practice LOs) have been applied in this new pandemic context; therefore, it is extremely important to investigate how they influence learning processes; therefore, it is necessary to know the perception of students regarding the significance of DLRs in the learning of statistics. As already indicated, validated instruments to assess students’ perception of the significance of DLRs in learning statistics are scarce. That is why Monge-Rogel et al. (2022) proposed the IPDLR-S instrument, which allows measuring the perception of students regarding the use of DLRs in the learning of statistics. An aspect to highlight of the instrument is that it was subject to a process of content validation, meaning that its items have a high degree of relevance and relevance to the objective of the construct (Haynes et al., 1995; Juárez-Hernández & Tobón, 2018; Koller et al., 2017).

According to the above, as referred to by Carvajal et al. (2011), the analysis of the psychometric properties of an instrument is a continuous process, and the greater the number of properties analyzed, the quality of the instrument is strengthened. In this sense, several authors consider that construct validity is the property of greatest relevance since it allows to confirm or adapt the theoretical construct of the instrument and provides reliability for the realization of new predictions (Brown, 2015; Houston, 2004; Wilson, 2010). In this sense, the construct validity analysis is carried out to guarantee the validity of the measurement; it relates the structure and demolishes that the theoretical and emerging relationships are appropriate (Lagunes-Córdoba, 2017; Mavrou, 2015). That is, construct validity guarantees that the instrument measures what it wants to measure since it uses valid measures of the studied constructs (Houston, 2004).

Therefore, in the present study, the construct validity of the IPDLR-S instrument was analyzed, the above through a cross-validation process, which is referred to as the process of greatest precision (Brown, 2015; Fokkema & Greiff, 2017; Schmitt, 2018).

In this regard, it is necessary to indicate that to perform the EFA, a polychoric matrix was used since the data originate from a Likert scale, which corresponds to variables that do not conform to a normal distribution (Watkins, 2020). According to Flora and Curran (2004), polychoric correlations tend to be robust to data violations.

According to the EFA, in the first instance, it is denoted that all the items are represented within the factorial model, which shows that the items represent the proposed construct (Lagunes-Córdoba, 2017; Mavrou, 2015). The foregoing reveals the importance of the content validation phase to which the instrument was submitted (Monge-Rogel et al., 2022), since content validity is an important step in construct validity because it contributes evidence regarding the degree to which the items of an instrument are relevant and pertinent to the construct (Haynes et al., 1995; Juárez-Hernández & Tobón, 2018).

The factorial model obtained through the EFA showed differences concerning the theoretical model since it went from three dimensions to four dimensions. This can be explained because the perception that students have regarding the functionality of DLRs when they ask for help or develop it with others (student or teacher) forms a new dimension. Regarding the migration or movements of items, it is specified that items 23, 29, and 30 were moved to the new Cooperative Learning dimension, which would be explained by what has already been mentioned around the cooperation with others regarding the use of the DLRs. According to the aspects addressed by these items, and given the current context of the COVID-19 pandemic, collaborative learning takes on a special connotation that, for researchers was not initially considered. This is explained since the DLRs were designed to complement student learning in statistics subject in collaboration with peers and teachers, learning situations that changed in emergency teaching (Kohnke et al., 2021; Xu et al., 2021). It is important to note that altogether the resulting factorial model explained 83% of the variance, which according to Hair et al. (2010) and Rietveld and van Hout (1993), is considered highly acceptable.

The evaluation of the resulting factorial model through the CFA revealed an optimal adjustment of the same, which is verified by the different indices and adjustment indicators used (GFI = 0.999, RMSEA = 0.064, CFI = 0.999, and TLI = 0.999). In this regard, through this analysis the empirical sustainability of the model (Hair et al., 2010) can be indicated meaning that the 4-factor model fits the instrument appropriately to measure students’ perception of the use of DLRs and their influencing factors.

Regarding the analysis of convergent validity of the factorial model, according to the indicators indicated (AVE > 0.50), it can be indicated that the constructs that are expected to be related, in fact, are (Fornell & Larcker, 1981). Regarding the discriminant validity, it was found that the correlations squared are less than the AVE of each corresponding factor; therefore, the model can discriminate between different constructs (Hair et al., 2010).

The global reliability (ordinal Cronbach’s alpha > .98), as well as by factors (ordinal Cronbach’s Alpha > .92), was optimal, meaning that the reliability of both the instrument and the factors is excellent (George & Mallery, 2003), that is, the instrument will produce consistent results, free of errors, always measuring the perception of the students in the use and functionality of the DLRs in the four dimensions that compose it.

Finally, it is important to highlight what refers to the affordability of the instrument to the target population; the understanding of the instructions and items was between good and excellent degree. The analysis of this characteristic is fundamental since it allows to ensure that the instrument and its items are understandable by the students and possible alterations of their answers by aspects of redaction and, at the same time, reduce possible problems with the items that may affect evaluative capacity (Carrillo-Avalos et al., 2020; Carvajal et al., 2011).

Although the IPDLR-S instrument applies to the DLRs that are used for teaching statistics, it is important to note that this instrument can be used for any subject. The foregoing is justified considering that the structure of the instrument allows making minimal changes in its wording to adapt the instrument to apply it to other subjects in science, technology, mathematics, or engineering.

One of the limitations of this study is that the designed instrument only covers higher education. Likewise, the validated instrument only allows evaluation of the perception of university students in the field of practical learning objects (Churchill, 2007).

Conclusions

It can be concluded that the IPDLR-S instrument (Monge-Rogel et al., 2022) has adequate psychometric properties, which is why it constitutes an innovative tool that will make it possible to reliably and validly evaluate the functionality of the RDAs from the student’s perspective.

Footnotes

Dataset

Monge-Rogel, Ricardo; Panes-Martínez, Mónica; Durán-González, Guillermo Bernardo; Juárez-Hernández, Luis Gibran (2022), “Student responses to the IPDLR-S instrument (Construct validity),”Mendeley Data, V1, doi: 10.17632/t4vjmjrhj8.1.

Declaration of Conflicting Interests

The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.

Funding

The author(s) disclosed receipt of the following financial support for the research, authorship, and/or publication of this article: This work was supported by the Universidad de Las Américas under Grant number 652019.

Ethical Approval

This study had the institutional approval of the Ethics Committee of the Universidad de Las Américas

Informed Consent

Informed consent was obtained from all participants who were included in this study.

ORCID iDs

Guillermo Durán-González

Mónica Panes-Martínez

Ricardo Monge-Rogel

Luis Gibran Juárez-Hernández

References

Adedokun-Shittu

N. A.

Shittu

A. J. K.

(2015). Assessing the impacts of ICT deployment in teaching and learning in higher education: Using ICT impact assessment model. Journal of Applied Research in Higher Education, 7(2), 180–193. https://doi.org/10.1108/JARHE-02-2013-0012

Ato

López

J. J.

Benavente

(2013). A classification system for research designs in psychology. Annals of Psychology, 29(3), 1038–1059. https://doi.org/10.6018/analesps.29.3.178511

Barattucci

(2017). Approach to study as an indicator of the quality of teaching and of learning environment: The contribution of John Biggs. Journal of E-Learning and Knowledge Society, 13(2), 77–88. https://doi.org/10.20368/1971-8829/1311

Basuhail

(2019). E-Learning objects designing approach for programming-based problem solving. International Journal of Technology in Education, 2(1), 32–41. https://www.ijte.net/index.php/ijte/about. www.ijte.net

Bhardwaj

(2019). Types of sampling in research. Journal of the Practice of Cardiovascular Sciences, 5(3), 157. https://doi.org/10.4103/JPCS.JPCS_62_19

Blunch

N. J

. (2012). Introduction to structural equation modeling using IBM SPSS statistics and AMOS. Introduction to structural equation modeling using IBM SPSS Statistics and AMOS, 1–312.

Bollen

K. A.

(1989). Measurement models: The relation between latent and observed variables (p. 2013). John Wiley & Sons. https://onlinelibrary.wiley.com/doi/abs/10.1002/9781118619179.ch6

Bollen

K. A.

Long

J. S.

(1993). Testing structural equation models. In Bollen

K. A.

Long

J. S.

(Eds.), Testing structural equation models (pp. 294–316). Sage.

Brown

T. A.

(2015). Confirmatory factor analysis for applied research (2nd ed). The Guilford Press.

10.

Carrillo-Avalos

B. A.

Sánchez Mendiola

Leenen

(2020). Threats to validity in evaluation: implications in medical education. Research in Medical Education, 34, 100–107. https://doi.org/10.22201/FACMED.20075057E.2020.34.221

11.

Carvajal

Centeno

Watson

Martínez

Sanz Rubiales

. (2011). How to validate a health measurement instrument? Annals of the Health System of Navarre, 34(1), 63–72.

12.

Centro Universitario. (2018). Cuestionario de satisfacción con el instrumento (plantilla).

13.

Chiu

T. K. F.

Churchill

(2016). Design of learning objects for concept learning: Effects of multimedia learning principles and an instructional approach. Interactive Learning Environments, 24(6), 1355–1370. https://doi.org/10.1080/10494820.2015.1006237

14.

Churchill

(2007). Towards a useful classification of learning objects. Educational Technology Research and Development, 55(5), 479–497. https://doi.org/10.1007/s11423-006-9000-y

15.

Deshpande

Shesh

(2021). Blended learning and analysis of factors affecting the Use of ICT in education. Advances in Intelligent Systems and Computing, 1162, 311–324. https://doi.org/10.1007/978-981-15-4851-2_33

16.

DeVellis

R. F.

(2016). Scale development: Theory and applications (p. 247). Book. https://books.google.com.pk/books/about/Scale_Development.html?id=48ACCwAAQBAJ&redir_esc=y

17.

Flora

D. B.

Curran

P. J.

(2004). An empirical evaluation of alternative methods of estimation for confirmatory factor analysis with ordinal data. Psychological Methods, 9(4), 466–491. https://doi.org/10.1037/1082-989X.9.4.466

18.

Fokkema

M. y

Greiff

(2017). How performing PCA and CFA on the same data equals trouble. European Journal of Psychological Assessment, 33(6), 399–402. https://doi.org/10.1027/1015-5759/a000460

19.

Fornell

Larcker

D. F.

(1981). Structural equation models with unobservable variables and measurement error: Algebra and statistics (vol.18, Issue 3, pp. 382–388). Sage. https://doi.org/10.1177/002224378101800313

20.

Furr

R. M.

(2020). Psychometrics in clinical psychological research. In Wright

Hallquist

(Eds.), The Cambridge handbook of research methods in clinical psychology (pp. 54–65). Cambridge University Press. https://doi.org/10.1017/9781316995808.008

21.

Garay Ruiz

Tejada Garitano

Castano Garrido

. (2017). Perceptions of the student to learning through enriched educational objects with augmented reality. EDMETIC, 6(1), 145–164.

22.

George

Mallery

(2003). SPSS for windows step by step: A simple guide and reference, 11.0 update (4th ed., pp. 1–400). Allyn & Bacon.

23.

Hair

Black

Babin

Anderson

(2010). Multivariate data analysis (7th ed.). N. P. H. Upper Saddle River.

24.

Haynes

S. N.

Richard

Kubany

E. S.

(1995). Content validity in psychological assessment: A functional approach to concepts and methods. Psychological Assessment, 7(3), 238.

25.

Hortense

F. T. P.

Bergerot

C. D.

Domenico

E.B. L. de

. (2018). Construction and validation of clinical contents for development of learning objects. Revista Brasileira de Enfermagem, 71(2), 306–313. https://doi.org/10.1590/0034-7167-2016-0622

26.

Houston

M. B.

(2004). Assessing the validity of secondary data proxies for marketing constructs. Journal of Business Research, 57(2), 154–161. https://doi.org/10.1016/S0148-2963(01)00299-5

27.

Howard

M. C.

Jayne

B. S.

(2015). An analysis of more than 1,400 articles, 900 scales, and 17 years of research: The state of scales in cyberpsychology, behavior, and social networking. Cyberpsychology, Behavior, and Social Networking, 18(3), 181–187. https://doi.org/10.1089/CYBER.2014.0418

28.

Jamalzadeh

Lotfi

A. R.

Rostami

(2021). Assessing the validity of an IAU General English Achievement Test through hybridizing differential item functioning and differential distractor functioning. Language Testing in Asia, 11(1), 1–17. https://doi.org/10.1186/S40468-021-00124-7

29.

Juárez-Hernández

L. G.

(2018). Manual práctico de estadística básica para la investigación. Kresearch. https://scholar.google.com/scholar?cluster=3526747967362827937&hl=en&oi=scholarr

30.

Juárez-Hernández

L. G.

Tobón

(2018). Análisis de los elementos implícitos en la validación de contenido de un instrumento de investigación. Espacios, 39(53), 23–30. http://www.revistaespacios.com/cited2017/cited2017-23.pdf

31.

Kay

R. H.

Knaack

(2009). Assessing learning, quality and engagement in learning objects: The Learning Object Evaluation Scale for Students (LOES-S). Educational Technology Research and Development, 57(2), 147–168. https://doi.org/10.1007/s11423-008-9094-5

32.

Kline

(1986). A handbook of test construction: Introduction to psychometric design. Methuen.

33.

Kohnke

Zou

Zhang

(2021). Pre-service teachers’ perceptions of emotions and self-regulatory learning in emergency remote learning. Sustainability, 13(13), 7111. https://doi.org/10.3390/SU13137111

34.

Koller

Levenson

M. R.

Glück

(2017). What do you think you are measuring? A mixed-methods procedure for assessing the content validity of test items and theory-based scaling. Frontiers in Psychology, 8, 126. https://doi.org/10.3389/fpsyg.2017.00126

35.

Lagunes-Córdoba

(2017). Recommendations on the procedures for the construction and validation of instruments and measurement scales in health psychology. Psychology and Health, 27(1), 5–18. https://psicologiaysalud.uv.mx/index.php/psicysalud/article/view/2431

36.

Lloret

Ferreres

Tomás

A. H. e. I

. (2017). Exploratory factor analysis of items: Analysis guided by empirical data and software. Annals of Psychology, 33(2), 417–432. https://doi.org/10.6018/analesps.33.2.270211

37.

Lüdecke

Ben-Shachar

Patil

Makowski

(2020). Extracting, computing and exploring the parameters of statistical models using R. Journal of Open Source Software, 5(53), 2445. https://doi.org/10.21105/JOSS.02445

38.

Mardia

K. V.

(1970). Measures of multivariate skewness and kurtosis with applications. Biometrika, 57(3), 519. https://doi.org/10.2307/2334770

39.

Mavrou

(2015). Exploratory factor analysis: Conceptual and methodological issues. Revista Nebrija de Lingüística Aplicada a la Enseñanza de Lenguas, 19, 71–80. https://doi.org/10.26378/RNLAEL019283

40.

Monge-Rogel

Durán-González

Panes-Martínez

Juárez-Hernández

L. G.

(2022). Design of an instrument to assess students’ perception of learning objects in statistics. Education and Information Technologies, 27, 9523–9539. https://doi.org/10.1007/s10639-022-11011-w

41.

Qiu

Dai

Yan

. (2022). Health behaviors of late adolescents in China: Scale development and preliminary validation. Frontiers in Psychology, 13, 1004364.

42.

Ráczová

Kačmár

Hricová

(2021). Psychometric evaluation and initial validation of the Slovak version of the Goal Adjustment Scale (GAS). Studia Psychologica, 63(1), 94–109. https://doi.org/10.31577/sp.2021.01.816

43.

Revelle

(2017). psych: Procedures for personality and psychological research. https://www.scholars.northwestern.edu/en/publications/psych-procedures-for-personality-and-psychological-research

44.

Rietveld

Van Hout

(1993). Statistical techniques for the study of language and language behaviour. Mouton de Gruyter.

45.

Rosseel

(2012). Lavaan: An R package for structural equation modeling. Journal of Statistical Software, 48(1), 1–36. https://doi.org/10.18637/JSS.V048.I02

46.

Salajan

F. D.

Perschbacher

Cash

Talwar

El-Badrawy

Mount

G. J.

(2009). Learning with web-based interactive objects: An investigation into student perceptions of effectiveness. Computers and Education, 53(3), 632–643. https://doi.org/10.1016/j.compedu.2009.04.006

47.

Sánchez

C. C. S. C

. (2019). The arrival of new technologies in education and its implications. International Journal of New Education, 2(4), 37–57. https://doi.org/10.24310/IJNE2.2.2019.7449

48.

Schmitt

T. A.

Sass

D. A.

Chappelle

Thompson

(2018). Selecting the “best” factor structure and moving measurement validation forward: An illustration. Journal of Personality Assessment, 100(4), 345–362. https://doi.org/10.1080/00223891.2018.1449116

49.

Schumacker

Lomax

(2016). A beginner. s guide to structural equation modeling (electronic version). Lawrence Erlbaum.

50.

Tabachnick

B. G.

Fidell

L. S.

(2001). Using multivariate statistics (4th ed., p. 966). Allyn and Bacon.

51.

Thompson

. (2004). Exploratory and confirmatory factor analysis: Understanding concepts and applications. American Psychological Association.

52.

Watkins

M. W.

(2020). A step-by-step guide to exploratory factor analysis with R and RStudio. https://doi.org/10.4324/9781003120001/STEP-STEP-GUIDE-EXPLORATORY-FACTOR-ANALYSIS-RSTUDIO-MARLEY-WATKINS

53.

Wilson

(2010). Essentials of business research—A guide to doing your research project (vol.12, Issue 2). Sage. https://doi.org/10.1177/097215091101200211

54.

Xia

Yang

(2019). RMSEA, CFI, and TLI in structural equation modeling with ordered categorical data: The story they tell depends on the estimation methods. Behavior Research Methods, 51(1), 409–428. https://doi.org/10.3758/s13428-018-1055-2

55.

Jin

Deifell

Angus

(2021). Chinese character instruction online: A technology acceptance perspective in emergency remote teaching. System, 100, 102542. https://doi.org/10.1016/J.SYSTEM.2021.102542

56.

Yong

A. G.

Pearce

(2013). A beginner’s guide to factor analysis: Focusing on exploratory factor analysis. Tutorials in Quantitative Methods for Psychology, 9(2), 79–94. https://doi.org/10.20982/TQMP.09.2.P079

Psychometric Properties of an Instrument to Evaluate Students’ Perception of Learning Objects in Statistics

Abstract

Keywords

Introduction

Methodology

Type of Study

Instrument

Selection of a Sample of the Population for the Application of the Instrument

Construct Validity and Reliability Analysis

Item Analysis

Factor Analysis

Exploratory Factor Analysis

Confirmatory Factor Analysis

Divergent and Discriminant Validity

Instrument Reliability and Composite Reliability

Instrument Satisfaction Analysis

Results

Construct Validity and Reliability Analysis

Construct Validity and Reliability Analysis

Factor Analysis

Exploratory Factor Analysis

Confirmatory Factor Analysis

Convergent and discriminant validity

Instrument Reliability and Composite Reliability

Instrument satisfaction analysis

Discussion

Conclusions

Footnotes

Dataset

Declaration of Conflicting Interests

Funding

Ethical Approval

Informed Consent

ORCID iDs

References