Abstract
This article questions whether there is a conflict between the academic and business interests of a university in addressing grade inflation. A survey of online master’s degree students at a British university was carried out. It asked students about a number of possible changes aimed at reducing the sacrifices involved in gaining a degree; making it less likely that students would fail modules or their degree; and reducing the degree drop-out rate to almost zero. Changes in the perceived value of a degree or the university’s perceived reputation were measured. The conclusion is that students saw such changes as reducing the value of their degree. The research suggests that in certain circumstances, academic and business interests can be aligned in opposing grade inflation.
Introduction
Many university teachers may often feel that the pressure in favor of grade inflation and a lowering of standards is relentless. They may be tempted to identify such pressure as being exerted by the business interests of their university, driven in turn by student demands to make courses easier. The view of Sadler (2009) is that “grade inflation occurs when high grades are awarded for progressively lower and lower achievements” (p. 823). It also implies that some students, who would not otherwise do so, will pass assessments. He argues for grade integrity, which is defined as the extent to which each grade awarded, either at the conclusion of a course or module of study, or for an extended response to an assessment task, is strictly commensurate with the quality, breadth and depth of a student’s performance. (Sadler, 2009, p. 807)
A number of harmful consequences of grade inflation have been identified. It leads to a compression of grades, making it difficult for employers to distinguish between students of different abilities. Different degrees of grade inflation lead to uneven, and hence unfair, assessment (Abbott, 2008). The credibility of grades can be undermined so that employers turn to other methods of evaluating job applicants (Wongsurawat, 2009). In terms of the theories of Spence (2002), the fact that a graduate owns a degree at a particular grade would be sending a misleading signal to potential employers.
It is understandable in such situations to regard teaching staff as defending academic standards on largely ethical grounds in opposition to business interests, which would be served by easier degree courses. This would be unfortunate. By broadening the perspective and looking at some of the economic forces that influence demand for degrees, it can be seen that well-established and validated marketing theory does not suggest that there will always be this conflict between academic and business interests.
To test this theory in the context of online education, we conducted a survey aimed at online students who were just about to complete their online master’s degrees offered by the University of Liverpool. The participants were asked about the effect on the perceived value of their degrees of a number of changes which were directly or indirectly associated with grade inflation. These changes were generally seen as reducing the value of their degrees, thus potentially harming the business interests of the university.
Theory
Pricing and the Sacrifice
Kotler and Armstrong (2010), in their classic work, Principles of Marketing, state that “consumers usually perceive higher-priced products as having higher quality” (p. 317). This implies that the cost of goods can be too low as well as too high. Teas and Agarwal (2000) write about extrinsic product cues, such as price and brand name, affecting in a positive way perceptions of quality. André Gabor, one of the pioneers in the field of research about price as an indicator of quality, gives examples of products that increased their sales when they put up their prices. He maintains that such judgments about quality are based more on consumers’ experience than on their ignorance.
Gabor (1988) concludes that the customer bent on a purchase will approach the market with two price limits in mind: an upper limit beyond which he would consider the item too expensive for his purposes. And a lower limit below which he could not trust the quality (p.254).
Between this upper and lower limit, price will not act as an absolute barrier to purchase (Gabor & Granger, 1966). This builds on earlier insights by French researchers whose work is not quite as accessible to those working in the English language (Adam, 1958; Stoetzel, 1954).
Marketing theory is flexible enough to realize that there are costs that are not related to financial price involved in the acquisition of goods and services. These total perceived costs, financial and non-financial, are often referred to as the give or sacrifice (Zeithaml, 1988). The latter term is particularly pertinent to university students taking a degree. The stress involved in balancing competing demands on time can be considerable. The suffering involved in acquiring a degree is particularly acute among part-time mature students who have so many more commitments than their younger full-time counterparts. These are the kind of students who are taught on completely online master’s degree programs run by Laureate Online Education on behalf of the University of Liverpool. The university is believed to be the biggest provider of online master’s degrees in Europe. The students have an average age of around 38, and are drawn from all parts of the world.
The current authors’ research focused on the question of whether it is possible that, in certain circumstances, degrees that involve greater sacrifice are perceived as having more value. Conversely, those degrees which involve few sacrifices could be seen as being of poor quality. If this is so, then there may well be an alignment of academic and business interests. The teacher who protects academic standards could be maintaining the value and economic desirability of the product.
Perceived Value
Alves (2010) conducted a wide ranging review of the literature on perceived value in higher education. Sancjez-Fernandez and Iniesta-Bonillo (2007) performed a similar service in looking at perceived value in general. Some of the literature treats perceived value as the net value of a product after all the pluses and minuses have been added up. In the definition of Zeithaml (1988), perceived value is “the consumer’s overall assessment of the utility of a product based on perceptions of what is received and what is given” (p. 14). The weakness of this is that it hides much of the detail. In particular, it does not tell us whether a high level of sacrifice is perceived as a plus or a minus. Too often, the sacrifice is seen as unambiguously negative, ignoring the obvious example of luxury goods where a high price is often seen as a positive feature.
Adjuncts
The term adjunct is peculiar to certain countries, particularly the United States, and it is not in widespread use in the United Kingdom. It usually refers to a part-time teacher without security of employment. A number of studies have suggested that the greater use of adjuncts, particularly in online classes, has contributed to the increase in grade inflation. Because there is no convenient British equivalent, the term adjunct will be used here.
If grade inflation is the result of inexorable pressures on degrees that rely on the use of adjuncts, then this would tend to undermine the argument in this article. This is because the University of Liverpool qualifications delivered by Laureate Online Education are taught almost exclusively by adjuncts. The evidence must therefore be considered.
Kezim, Pariseau, and Quinn (2005) looked at grades at an American college over a 20-year period and concluded that adjuncts contributed to additional grade inflation. A popular explanation of this phenomenon is that adjuncts are judged largely or exclusively on student evaluations, and higher grades are a means of trying to increase popularity. The insecurity of employment experienced by adjuncts is suspected of increasing this desire to curry favor.
Ewing (2011) found there was a positive correlation between students’ expected grades and their evaluations of teachers. The study of Love and Kotchen (2010) concludes that “increased institutional emphasis on teaching evaluations can exacerbate the problems of grade inflation and inadvertently lower faculty teaching effort” (p. 162).
The research of Langbein (2008) discovered that actual and expected grades had an effect on student evaluations of teaching. Because of the widespread use of such evaluation methods in the United States, she feared that they would fuel universal grade inflation, although this would be particularly acute in private colleges. Eiszler’s (2002) examination of more than 900,000 student evaluations concludes that they were significantly related to expected grades. Crumbley, Flinn, and Reichelt (2010) feel that student evaluations encourage a range of unethical behavior in teachers aimed at improving ratings without increasing learning.
Bar, Kadiyali, and Zussman’s (2009) study reports that “in the spring semester of 1998, Cornell University started publishing median course grades on the Internet. Our analysis finds that the provision of grade information online induced students to select leniently graded courses” (p. 107).
In assessing the impact of such practices on adjuncts, Sonner (2000) finds that adjunct instructors, keen to improve their evaluation results, did give higher grades than full-time teachers. Cavanaugh’s (2006) research suggests that a typical student would receive a B grade from an adjunct and a C from a full-time member of staff. It is not perhaps an accident that most of the evidence for grade inflation is coming from those who teach undergraduates in the United States.
The degrees which are featured in this current article are normally taught by teachers who are all adjuncts. Their module managers, faculty managers, and directors of online studies are also adjuncts. A large number of these come from the United States, where grade inflation might well be a daily reality of their teaching on campus. All online teachers are subject to student evaluations at the end of every module. At the same time, there has, to date, been a strong culture of opposing grade inflation.
The degree programs described in this article apply very strict grading criteria. Only around 20% of the students could expect to receive A or A* grades. This can be contrasted with the case at Princeton University, which reported as a success the fact that the proportion of A+, A, and A− grades on undergraduate courses had fallen from 47.9% in 2002-2003 to 39.7% in 2008-2009 (Quinones, 2009). International students are generally advised that a Grade C at Liverpool is equivalent to Grade B in the United States. There is no internal evidence that grades have increased over the decade in which the oldest of the University of Liverpool online degrees have been provided. How is that possible?
The answer is that instructors can normally gain no advantage by providing higher grades to students in the absence of evidence that work of superior quality has been produced. Although student evaluations are used, they constitute just one of many indicators of performance for online teachers. Alongside such evidence, a faculty manager will consider whether the grading profile of a teacher is significantly different from his or her peers. It would certainly be considered undesirable if popularity had been bought, intentionally or otherwise, by the offering of higher grades. Thus, a teacher demonstrates that he or she is valuable to the university partly by showing that he or she does not award higher grades without justification.
The conclusion is that there is nothing inevitable about grade inflation in circumstances where there is an intensive use of adjuncts subject to student evaluations. The negative consequences are purely the result of specific academic or business decisions, which in turn seem to be the result of very poor management. If a university chooses to make appointment and promotion decisions based solely or predominantly on student evaluations, particularly in the face of evidence that undergraduates respond to those who grade generously, then it should expect to be embarrassed.
In the world of online education, where all interactions can be stored in a virtual learning environment, there is no excuse for lazy management. A broad range of data is typically available in an electronic form so that more rounded evaluations of teachers can take place. Teaching in a goldfish bowl, where teachers realize that everything they have ever written to any student in the last 10 years is available to external regulatory bodies such as the Quality Assurance Agency for Higher Education (QAA), is not always comfortable. It does, however, provide relatively unambiguous evidence of who did and said what to whom. Unjustified criticisms of instructors can be refuted by documentary evidence, and teaching staff are less reliant on receiving positive feedback in circumstances where difficult decisions have to be made.
Oleinik (2009) pessimistically concludes that “grade inflation results from the substitution of criteria specific to the search for truth by criteria of quality control generated outside of academia” (p. 156). He did not foresee that certain systems of quality control would be aimed at preventing grade inflation on the understanding that courses with inflated grades are of reduced quality.
Areas for Investigation
In this study, the approach was to isolate a number of variables to see whether something interesting, and possibly unexpected, could be identified. A cluster of three issues was identified, which might be seen as associated with grade inflation and a lowering of academic standards.
The level of sacrifice involved in obtaining a student’s degree. We asked respondents to ignore the cost of fees and concentrate on other sacrifices they had made. The question of tuition fee costs as a sign of quality was tested separately.
The proportion of students who failed their degree or individual modules (a module is an intensive class lasting 8 weeks). The survey asked about the case where the number of failures was very low. It was reasoned that where grades were uniformly inflated, the number of failures would fall, and that questions such as this would be the easiest way of testing attitudes toward grade inflation.
The numbers of students who dropped out of their degree. Ormond Simpson (2003), in a book based on his experiences at the Open University in the United Kingdom, expresses the opinion that “an institution that awarded its own qualifications and had zero dropout might have difficulty in persuading both its students and the general public that its qualifications were worth having” (p. 11). Part of the current authors’ research was aimed at establishing how far this was true.
The responses to some of the questions would probably depend on the current situation. The reaction of students could be different if they were enrolled on a degree course where grades were already inflated. As has been noted, the baseline position was that grade inflation was not a problem, and students would therefore be expected to work hard for good grades. It could not be considered easy to pass the degrees in question. This is consistent with what colleagues in marketing would call the positioning of the degrees.
Two authors who pioneered the programs, Kalman and Leng (2007), write about “the widespread public suspicion that this educational [online] paradigm will be associated with low standards and quality.” They concluded that, in overcoming this perception, “quality cannot be bought cheaply” (p.59). Computing colleagues, Grasso and Leng (2005), state that “we believe firmly that the successful programmes of the future will be those that focus on pedagogy, and give precedence to academic standards and quality assurance, rather than those that emphasize technological aspects or focus on low-cost delivery” (p. 2386). This meets the concerns expressed by Thirunarayanan (2001) a decade ago that online education might lead to people receiving degrees who did not really deserve them.
A decision was taken to gather students’ views through the medium of an anonymous online survey. What the survey sought to measure was perceived value. Students were not asked to apply a range of objective criteria in making their judgments. In general, they were simply asked whether certain things increased or decreased the value or quality of their degree in their eyes.
Method
An online survey was carried out in 60 small Research Methods classes of the University of Liverpool’s online master’s degree programs. The main subjects were computing, business administration, and management, with smaller numbers from public health and clinical research administration. Research Methods is the module students take toward the end of their degree immediately before attempting their dissertation, and after completing 8 other modules (a module is an 8-week class). Some 369 students completed the survey anonymously with a response rate of 44.8%.
One of the concerns of using a publicly visible online survey is that, either through misplaced enthusiasm or malice, the web address could be made available to those outside the target audience. There is then a danger that the sample could be corrupted by the presence of large numbers of respondents whom the researchers did not want to take part.
To reduce the risks of this happening, and to limit the damage if it did, the authors used what they called, somewhat tongue in cheek, the New Titanic Method. It will be recalled that the RMS Titanic was meant to have a number of watertight compartments which would help it stay afloat if the hull was breached. In practice, the compartments were not watertight, and the damage inflicted by a collision with an iceberg was so severe that the vessel sank.
The response was to divide the online survey into a number of compartments that were a little more watertight than those on the Titanic. For each of the 60 classes that were surveyed, a blackboard message (internal email) was sent to every member of a particular class inviting them to take part in the survey. Each class was directed to a separate web address.
The service that was used, SurveyGizmo, made it easy to generate new instances of the survey, each of which had its own web address. Hence, there were 60 copies of the survey residing at 60 web addresses, and generating 60 separate data sets. These data sets were combined during the analysis.
The consequence of adopting this approach was that if a single web address was leaked to a wider audience, only one compartment in our survey ship would be flooded by invalid data. It could then be disregarded without corrupting the whole survey. In the event, there is no evidence that such a leak occurred. The numbers responding to the survey never exceeded, or even approached, the number of students in each class.
Many respondents did not have English as their first language, and so they were asked to indicate whether any question in the survey was so difficult for them to understand that their reply might not reflect their true opinion. Where a student indicated that a question fell into that category, the response to that question was discounted.
Students were asked for their age and gender, and the survey service that was used, SurveyGizmo, identified their location from their IP address. Students were asked about the actual sacrifices they had made compared with the sacrifices they expected to make; how important it was to them that their degree was being awarded by the University of Liverpool; their level of satisfaction with their degree course; and the survey asked about their views on the relationship between tuition fees and course quality.
The core of the survey addressed questions of perceived value in situations where changes were made that would generally make the course easier. These were grouped as follows:
Questions that asked about the effect on the perceived value of the degree if, shortly after the student’s graduation, changes were made that resulted in modules being much easier to pass, students being unlikely to drop out of their degree because it was too difficult, or students finding that gaining a degree involved far fewer sacrifices.
Questions about the effect on the perceived value of the degree if it involved very little non-monetary sacrifice, students hardly ever failed, or the drop-out rate was close to zero.
Questions about the effect on the University of Liverpool’s reputation if students rarely failed modules or rarely dropped out because the work was too hard.
For the most part, semantic differential questions were used, with possible options ranging from 7 (increased value/higher reputation) to 1 (decreased value/lower reputation). This has been treated as interval data for which the mean score can legitimately be calculated. The choice by participants of the mid-point value 4 was assumed to indicate a degree of neutrality or indifference toward the issue in question; scores of 5 to 7 were interpreted as being broadly positive, and scores of 1 to 3 negative. The results which were consistent with the authors’ expectations would involve a mixture of high and low scores.
Results
Sacrifice and Tuition Fees
Some 79% of students indicated that their degree had involved greater sacrifices than they had expected. Only 5% thought it involved less sacrifices. The mean score was 5.5. At the same time, 88% of the sample were satisfied with their degree (M = 5.7) and 88% would recommend their degree to others (M = 5.9). It can therefore be seen that there is no incompatibility between high student satisfaction and a higher than expected degree of sacrifice.
The assumption had been that high tuition fees would not be a strong indicator of course quality, and that other aspects of the give or sacrifice involved in obtaining a degree would be more important. This was confirmed by the survey. When asked whether courses with high tuition fees were typically of better quality than those with lower tuition fees, respondents produced a mean score of 3.8 (median = 4). The numbers of positive, neutral, and negative responses were almost precisely equal.
Changes to Degrees After Graduation
Possibly the most significant results from the survey were produced by the questions that asked about future changes that would make degrees easier in certain specified respects after the student had graduated. The general conclusion was that students would see such changes as devaluing their degree. This would be likely to generate significant negative word-of-mouth (as well as negative social network and email traffic) about their degree program.
If, in future, it would be much easier to pass each module on the degree, 69% of the respondents felt that this would decrease the value of their degree and only 14% believed that its value would increase (M = 2.8, median = 3). Similarly, 66% saw a future degree that would involve far fewer sacrifices on the part of students as involving a lowering of value. Just 12% of the sample thought this would increase the degree’s value (M = 3.0, median = 3).
Throughout the survey, respondents were less concerned about low drop-out rates as a negative indictor, but 52% of the sample believed that a future in which students would be unlikely to drop out of their degree because it was too difficult would signal a qualification that had less value. Twenty-five percent took the contrary view (M = 3.4, median = 3). The mean scores for these three questions are highlighted in Figure 1. The percentage of respondents who thought these changes would be positive or negative is represented in Figure 2.
Mean values for the assessment of future changes.
Percentages of those who saw changes as having a positive or negative effect on the value of the degree.
Less Sacrifice, Low Failure/Drop-Out Rates
The next set of questions involved subtle differences in wording. Instead of being asked about failing modules, students were quizzed about their view if people hardly ever failed their degree. Fifty-eight percent of the respondents felt this would decrease the value of the qualification. This represented a slight softening of attitudes, because degree failure was clearly a harsher penalty than module failure, and hence a higher price to pay for degree quality. However, the polarity of student opinion was clear, with only 14% feeling the degree would increase in value (M = 3.2, median = 3).
When asked how they would rate their degree if it involved very little sacrifice on their part, 60% said their qualification would decrease in value, with 15% taking the contrary view (M = 3.1, median = 3). A drop-out rate close to zero was seen as a negative thing by 46% of the respondents, whereas 23% saw it as positive (M = 3.6, median = 4). In this case, the reason for dropping out was not mentioned in the question. A summary of the relevant percentages is set out in Figure 3.
Percentages of those who saw features as having a positive or negative effect on the value of the degree.
The University’s Reputation
The University of Liverpool brand proved to have been an important factor in attracting students to their degree program. Many online degrees are offered by institutions whose names are not immediately recognizable. Liverpool is one of the older British civic or redbrick universities formed around the turn of the 20th century. It is a member of the Russell Group, a collection of research-intensive universities in the United Kingdom.
In the survey, 85% of the students indicated that the fact that the University of Liverpool was offering their degree was important or very important. Only 6% thought this factor was unimportant (M = 5.9, median = 6). Students were therefore asked about the effect of certain scenarios on the Liverpool brand. The result of students rarely failing modules on their degree course would, in the opinion of 49% of the respondents, lower the reputation of the university (M = 3.5, median = 4).
A very significant 29% of the sample opted for the mid-point value on the scale, indicating some degree of indifference. If that assumption of indifference is correct, then the natural conclusion is that the university does not need to worry about the reaction of a certain proportion of its students to modules getting easier. The reaction is likely to be neither positive nor negative. The focus will therefore switch to the proportions of students who react positively (22% in this case) and negatively (49%). On balance, therefore, it was concluded that modules that are rarely failed would damage the reputation of the university.
The sample was also asked about the effect on the University of Liverpool reputation if students rarely dropped out because the work was too hard. Here the results were inconclusive. The split meant that 35% thought this change would lower the reputation of the university, 36% thought it would give Liverpool a higher reputation, and 29% were indifferent (M = 4.0, median = 4).
Although the question did specify a particular reason why students might drop out of degree courses, informal feedback suggested some participants might have taken account of other reasons, such as illness or a change in personal circumstances. The retention of students under these difficult circumstances would be seen as a positive sign. It was also likely that the reputation of the university was considered so robust that changes were less likely to damage its image.
It was concluded that the use of drop-out rates as a signal of perceived value was problematical and any future research would require more incisive questioning to separate out the various factors involved.
Demographics
Gender, geography, and age
Just 33% of the survey sample was female. There were no statistically significant differences (p < .05) with the one- or two-tailed test in the answers provided by men and women.
In analyzing the sample according to the part of the world in which students were resident, the only significant differences were between the 21% of students who came from Africa and the rest. The results of this analysis are presented in Table 1. African students were more likely to believe that the sacrifices they had had to make were greater than expected (p = .0001) and to value the Liverpool brand in choosing their degree (p = .00003). They were less likely to believe that future changes after their graduation that would make their degrees easier would reduce the value of those degrees. In the case where it would be much easier to pass each module, p = .002; where it would be unlikely for students to drop out because the degree was too difficult, p = .0007; where gaining a degree in the future would require far fewer sacrifices, p = .01. African students were more satisfied with their degree (p = .0000005) and more likely to recommend it to someone else (p = .002). In almost all cases, the polarity of African and non-African students’ views was the same, meaning they were both positive or negative in relation to any particular question, but one group’s attitudes were stronger to a statistically significant degree. The two questions where there was an opposite polarity were the above mentioned case about future changes where students would be less likely to drop out (African M = 4.1, non-African M = 3.3), and the question that asked about the effect on Liverpool’s reputation if students rarely dropped out (African M = 4.5, non-African M = 3.9).
Comparison of African Students With the Rest of the Sample.
An examination of correlation results showed that the relationship between age and other variables was almost completely random. Although age was not generally a determinant of outcomes, there were a few statistically significant differences when comparing students who were above 40 (38% of the sample) with the rest (the mean age of students was 38). The older group felt in Q4 that the damage to the university’s reputation where students rarely dropped out would be greater than it was in the eyes of younger students (p = .04). Similarly, the above 40s were in Q8 more concerned by future changes that would make it easier to pass each module (p = .02). They were a little more inclined in Q13 to believe that degrees with high tuition fees are of better quality (p = .04). In Table 2, the lower scores indicate a lower perceived value or greater damage to the university’s reputation.
Comparison of Students Above 40 With the Rest of the Sample.
Degree comparisons
In comparing the results across degree programs, the most noticeable feature was that there were relatively few statistically significant differences. To isolate any variations that were peculiar to a particular degree program, a series of t test comparisons were conducted in which those in Degree X were compared with a combined group consisting of those not in Degree X.
Computing students felt that they had made slightly fewer sacrifices to obtain their degree than other students (p = .006). With respect to future changes after graduation that would make it easier to pass each module (p = .04) and make it unlikely that students would drop out because their degree was too difficult (p = .03), computing students were a little more likely to see these changes as negative. The level of module failure (p = .0004) and degree dropout (p = .008) at which students could not recommend their degree to others was higher than among non-computing students.
Management students were studying for an MBA or an MSc in management. These two groups did not display statistically significant differences, except in two less crucial areas—the latter felt the Liverpool brand was more important in choosing their degree, and they were more satisfied with their program. They were therefore treated as one group.
In a kind of mirror image of the computing students, the respondents taking management degrees felt they had made greater sacrifices than expected (p = .0008). In choosing a level at which the module failure rate was too high to recommend their degree to others, management students named a lower figure than other respondents (p = .006).
Clinical research administration students were a little less likely to recommend their degree to others (p = .005). In other respects, they were very similar in their views to those of other students taking other degrees.
Public health students had a higher average age of 41. They made up just 10% of the total sample, and therefore, the comparisons should be treated with some caution because of the relatively small numbers involved. However, it was noted that public health students were rather less concerned about the effect of low drop-out rates on the value of their degree (p = .03) or on the reputation of the university (p = .05). They were similarly less bothered by the effect of lower module failure rates on the university’s reputation (p = .0006), to the extent that, on this question, they actually saw the change as slightly positive (M = 4.2). They were even more likely to recommend their degree to others (p = .04). The percentage at which module failure rates cast doubt on the quality of their degree is lower for public health students (p = .02), as is the percentage drop-out rate at which the degree could not be recommended to others (p = .03).
Not Enough Sacrifice?
The authors wanted to get an idea of when the level of student module failures and dropouts due to the difficulty of the degree was considered too high or too low. To answer these questions, use was made of some techniques employed in the Price Sensitivity Meter of the Dutch economist Peter van Westendorp. The typical use of the Price Sensitivity Meter is to establish when the price of a product or service is considered to be too high or too low by potential consumers. It often helps in setting the price of a new product or in establishing whether the current price of a product is appropriate (Lipovetsky, Magnan, & Zanetti-Polzi, 2011).
The approach of van Westendorp was adapted to the slightly different situation being addressed by ourselves. Whereas the original technique would ask when the price was too low for it to be of good quality, in this study, respondents were asked about the average percentage of students who fail each module:
In relation to the percentage of students who decide to withdraw from their degree course (drop out) before the dissertation stage because the degree is too hard, the corresponding question was as follows:
The application of the Price Sensitivity Meter would then normally involve asking respondents when the price was so high that they would not consider buying the product. The two questions that were asked in this survey to test the case where values were deemed to be too high were as follows:
In all cases, respondents were asked to select a value from a drop down menu. The values available were in a range from 0% to 100% with 5% increments. The very first option on the menu was “No figure is too low” in the case where we were asking about values that were too low and “No figure is too high” when we were asking about values that were too high. Thus, participants were given every opportunity to indicate that they were not concerned about this issue.
Although these questions were in a well-established format, and the wording was as clear as it could reasonably be made, the concepts involved were not necessarily easy to grasp. The authors therefore included an optional question at the end of the survey: Were any of our questions so difficult to understand that your reply might not reflect your true opinion?
For most of the survey questions, the numbers indicating this degree of difficulty in understanding a question were very small. In the case of Questions 14 to 17, it was more significant. The percentages indicating a degree of difficulty for these four questions were 14.1%, 10.8%, 16.5%, and 11.9%.
The analysis of the data erred on the side of caution by discounting the replies to questions where respondents had indicated that their reply might not reflect their true opinion. However, a statistical examination of the replies to Questions 14 to 17 of those who did and did not indicate this difficulty showed that the difference between the mean values was not statistically significant (p < .05) for any of the four questions. In terms of the median scores, the inclusion of respondents who had doubts about their reply reflecting their true opinion would not have changed the median at all in the first three questions. For Q17, the effect would have been to increase the median from 45 to 50. The conclusion was that the doubts that some respondents experienced had affected their responses in the most trivial of ways. The exclusion of those who had doubts did not affect the median or mean scores to any significant degree.
The right hand column in Table 3 has been interpreted as the proportion of students who were indifferent to module failure or drop-out rates in assessing degree quality (Q14 and Q16) or in deciding whether to recommend their degree to others (Q15 and Q17).
Students Who Were Indifferent to Low or High Failure or Drop-Out Rates.
The replies to Questions 15 and 17, which asked about when values were too high, were something of a surprise, but were consistent throughout the phased conduct of the survey. They indicated that the level of satisfaction students felt with their degree course was such that only when failure or drop-out rates were approaching 50% would a majority begin to stop recommending their degree to others in large numbers. The results did indicate a high tolerance of significant failure and drop-out rates on the part of the more retainable students.
In Questions 14 and 16, the authors have taken a cautious approach in drawing conclusions about minimum module failure and drop-out rates by focusing on the median values. It does seem clear that if these rates were to fall below 10%, then large numbers of students are likely to see this as indicating a reduction in degree quality. A reasonable interpretation of the data in Table 3 is that an average of 0% module failures would be interpreted as a reduction in degree quality by 87.5% of the students. A drop-out rate of 0% would be similarly interpreted by 84.1% of the participants.
Park and Choi (2009) outline a number of ways in which retention can be improved by legitimate means without grade inflation. There is, however, likely to be a level beyond which it is unwise to go if the perceived value of degrees is to be maintained.
In other questions, participants were asked about a drop-out rate that was close to zero, or where students rarely dropped out because the work was too hard. It was felt that asking about the situation where no-one dropped out would be tantamount to asking about a situation that would never, in practice, happen. Respondents would have probably not seen this as a credible scenario. Although those taking the survey might not have difficulty in assessing the effect on perceived value of drop-out rates that were close to zero, or situations where students rarely dropped out, there was always the danger that they would privately define those terms differently.
Q16, inspired by the Price Sensitivity Meter, showed that a large majority of students were able to nominate a drop-out rate at which the degree’s quality would be doubted, and just 15.9% rejected the notion that there was any such figure.
It does seem to be a requirement of students’ willingness to undergo a certain amount of suffering to obtain their degree that such hardship is not seen as unjust or unreasonable. The degree satisfaction rates revealed in this survey suggest that there is no such perception. Theoretically, this can be linked back to the work of André Gabor (1967), who maintained that a price (sacrifice) that was seen as neither too high nor too low by buyers and sellers could be regarded as a just price for a product or service.
The authors do acknowledge that where a university has a reputation for recruiting the most talented and motivated students in a particular country, such as is the case with the Universities of Oxford and Cambridge in the United Kingdom, the perceptions of what constitutes an acceptable minimum drop-out rate may well be much lower. In the absence of such exceptional factors, it has been concluded that there will routinely be minimum failure and drop-out rates below which it would be unwise for a university to fall. Degrees where everyone passes and nobody drops out because the work is too hard will typically be viewed as qualifications of poor quality.
Until around the 1960s, there was a reasonably widespread belief that a lower price was always better for the customer. This was not based on strong empirical evidence, and when empirical studies were carried out, it was soon discovered that prices could be too low as well as too high, and that this finding was not limited to the special case where rich buyers were trying to impress their friends (Gabor & Granger, 1966; Shapiro, 1968). Some 50 years later, we may be on the verge of acknowledging the business problems created by universities whose degrees are obtained at a level of suffering that is too low.
Correlation Analysis
Where students were satisfied with their degree, they would typically recommend it to others. A correlation of .76 between these two variables underlined this close relationship. The results of the correlation analysis are shown in Table 4.
Correlation Matrix.
The key questions in the survey, which asked about how changes or different scenarios would affect the quality of degrees or the reputation of the university, were Q2 to Q9.
In Questions 8, 9, and 10, respondents were asked to indicate whether the suggested change would increase or decrease the value of their degree. It will be recalled that in each case, students were presented with a 7-point semantic differential scale. See the Appendix for the wording of questions.
An analysis showed that there were a few questions where replies were quite well correlated (> .40) with other responses. A key question was the following:
Respondents were asked to select a point on a scale indicating whether the university would have a higher or lower reputation. The replies were well correlated with six out of seven similar questions, indicating that it was a reasonable predictor of other attitudes. Knowing the answer to this question would provide a fair indication of (correlation figures in brackets) the value of the student’s degree if people hardly ever failed (.59); its value if the drop-out rate was close to 0 (.46); the effect on the university’s reputation if students rarely dropped out because the work was too hard (.52); the respondent’s attitude if, in the future, each module was much easier to pass (.43); the attitude toward a future change where students would be unlikely to drop out because their degree is too difficult (.41); and the student’s attitude toward a degree that would in future require far fewer sacrifices (.45).
Establishing the relationship between concepts by mathematical means is sometimes a little like typing a letter wearing boxing gloves, but the mathematical relationships can be suggestive. For the authors, the cluster of correlations around Q6 pointed to the centrality of the Liverpool brand and to anything that would damage the university’s reputation.
It has already been seen that the brand was an important factor in students choosing their degree. The most natural interpretation of the data is that if a degree’s becoming easier is seen as damaging the brand, then similar changes (as described in other questions) would also be seen in a negative light. Only where the change was not seen as damaging the brand would it be viewed as having no harmful effect on the quality of a degree. This is also true of future changes that might be made after graduation. Such changes were likely to be seen as diluting the value of a Liverpool degree obtained at some point in the past.
The other area with the highest correlations was that relating to future changes after graduation. The correlations were Q8 to Q9, .54; Q8 to Q10, .48; and Q9 to Q10, .58. Students tended to see such changes as being distinctly all positive or all negative.
Discussion
Reduced Value
The results of the survey show that, at least in certain circumstances, students do perceive high pass rates, low drop-out rates, and degrees that involve very little sacrifice on their part as reducing the value of their qualification. This perception is particularly strong in the case where post-graduation changes are proposed that would make the degree easier to obtain. Students who have made great sacrifices to gain a master’s degree would be particularly unhappy at seeing their degree devalued.
The survey sample consisted of students who were survivors. Those who had dropped out (typically in the first or second module of the degree) could have a different view. However, the authors’ informed conjecture is that the support services offered to online students by Laureate and Liverpool are gradually eliminating avoidable dropouts. Those who leave are increasingly likely to be those who see how hard the online program is and realize that they cannot fit it into their schedule.
If this is true, then at least some of these early dropouts are possibly a necessary evil in proving that this example of online education is not an easy option. It would seem a far better business option to appeal to those who are similar to the people in the sample who had demonstrated that they were retainable. Their desire is for degrees that, through the sacrifices involved, indicate their high value.
The authors’ research highlighted a cluster of interesting variables. It is possible that other variables will have an effect on students’ attitudes toward degrees that are challenging. Further research in comparing the influence of different variables would be welcomed.
The high proportion of students who felt that their degree had involved greater sacrifices than they expected does suggest that, at the time of their enrolling for the degree, the marketing and other university literature did not prepare them satisfactorily for what was to come. This may have changed in more recent times.
This does point to the dangers of being unduly positive in promotional material. It can inflate avoidable drop-out rates. First, because those who have believed the myths about online education being an easy option do not have those myths sufficiently dispelled by what they read about their degree. This encourages applications from people who are not suited to the workload or the pedagogical/andragogical model used. Such people might not apply if promotional material consistently pointed out how difficult study was likely to be.
Second, being more realistic in public statements would reduce the number of avoidable dropouts among students who felt that they were failures because they found online education extremely challenging. A university can say, “This degree course is going to be very difficult, but ultimately rewarding. You will experience many problems in balancing all your commitments, but we have the support services that can help you.”
Such an approach would tend to engender different expectations, so that the student might think, “I have problems, but this is normal. It does not mean that I am inadequate.” It is also likely that there are strong potential students who have believed the stories about online education being an easy, low-quality option (which in some cases might be true). It is possible that more honest marketing, telling them that they will have to suffer to gain a degree, would actually have a positive appeal. It could open up new markets by astute product positioning. In terms of signaling theory, it could be said that the university would be sending a signal to potential buyers that its products were of high value (Spence, 2002).
The data suggest that retainable students taking the master’s degrees discussed in this article see factors that add to the difficulty in obtaining their qualification as increasing its perceived value. This will inevitably influence their word-of-mouth recommendations (and these days that includes recommendations made by electronic means of communication). In this case, there is strong evidence that academic and business interests are aligned. Pressures for grade inflation and a degree obtained with fewer sacrifices are likely to undermine the value of the product and make it less attractive in the eyes of retainable students.
A business manager who was asked “How difficult is it to resist the temptations of grade inflation?” could reasonably reply, “It is quite easy. Grade inflation would damage our business by making the product less attractive.”
The Potential Scope of These Findings
Under what circumstances are these factors likely to apply? What is the scope of these conclusions? Marketing theory suggests that when an organization is selling a high value, high price product, the high price can be a positive attraction. This is because it makes the product exclusive. The owner possesses something that others do not have.
In education, it is the total sacrifice involved in obtaining a degree, rather than the level of fees, that provides this exclusivity. Conversations between the authors and online students after graduation ceremonies suggest that it is precisely because students have suffered to gain their degree that they regard it as having a high value. Under no circumstances would they accept that studying online was an easy option.
Are all degrees likely to engender this feeling in their students? Almost certainly not. A master’s degree, particularly one from a reputable university (one with a strong brand) is still something that sets an employee apart from others. It does, in many circumstances, still provide a career advantage. It is exclusive because large numbers of people do not have a master’s degree. In such circumstances, the business pressures to combat grade inflation are likely to be strong. Although the amount of fees charged is not in itself crucial in indicating the quality or value of the degree, it is likely that students who perceive their degree as having a high value are more willing to pay high fees.
On the other hand, where the possession of certain kinds of degrees becomes commonplace (a bachelor’s degree in some countries, for instance), the product takes on more of the features of mass-produced consumer goods. Here there are more pressures to drive the price down. In education terms, the emphasis switches to the importance of the grade that is obtained, because the mere possession of a degree has a reduced value. The pressure here is often to reduce the amount of sacrifice involved in getting a good grade. Pressman (2007) would identify this as demand-pull inflation, where too many students are chasing too few jobs.
It is clearly much easier to prevent grade inflation that has not historically taken place than it is to reduce grades that have become inflated. Once the proportion of A grades has risen above 40%, it is difficult to get the genie back in the bottle. However, the business case for doing so is suggested by the results of this research. There is a market for high-value degrees, the acquisition of which involves a significant amount of suffering on the part of the student. Indeed, it is possible to argue for the necessity of suffering. The student who has suffered is likely to believe that he or she has obtained a prize of great value.
Where degree programs require a high level of student participation, such as in the numerous assessed asynchronous online discussions favored by Liverpool, a campaign against grade inflation might also be a good way of getting students to work harder and participate to the desired extent. Babcock (2009), in surveying a large number of students at the University of California, San Diego, concluded that holding fixed instructor and course, classes in which students expect higher grades are found to be classes in which students study significantly less. Results indicate that average study time would be about 50% lower in a class in which the average expected grade was an “A” than in the same course taught by the same instructor in which students expected a “C.” (p. 983)
Although the University of Liverpool degrees are delivered in conjunction with an American private sector organization, Laureate Online Education, the authors have noted no pressures for grade inflation from their paymasters in their day to day teaching. There is therefore little cost-push pressure for grade inflation where teachers are encouraged to give higher grades by their institution (Pressman, 2007). Why? Because it would be bad for business.
The pressure is likely to be in the opposite direction, particularly in the case of Americans and other instructors who might be operating in on-campus courses characterized by rampant grade inflation. When they become online teachers at Liverpool, they have to adjust quickly to their new environment.
On the demand side of the equation, Liverpool students may wish that the assignment they are currently completing were not so time-consuming. In the passing moment, they may wish for an easier life. However, in the longer term, they value the suffering they have had to undergo. This may not be universally true, but it would be surprising if there were not general lessons to be learned here. The sacrifice is transitory, but positive recommendations persist over the long term.
Ledden and Kalafatis (2010) highlight the fact that degrees can have a different perceived value at different times. Therefore, when we ask students about perceived value might be important. Asking students during a temporary period of suffering might give a distorted view of the long-term benefits of such experiences to students who will, in graduating, gain a prize of great value.
Conclusion
Pricing theory suggests that the price of a product can be too low as well as too high. That price can include non-financial contributions the buyer has to make in acquiring the product. It can, for instance, include the sacrifices the student makes in acquiring a degree. The greater the sacrifice, the greater the perceived value of the product can be.
It therefore does not follow from this body of theory that there are inexorable pressures to reduce student sacrifices by making degrees easier to obtain. There are theoretical grounds for believing that, at least in certain circumstances, a high level of sacrifice will increase the perceived value of a degree. This suggests that there are some situations where the ethical demand of teachers to maintain academic standards could be quite compatible with the business goals of a university in maintaining the value of the product it sells.
To test the applicability of this body of theory to the online degree programs of a prominent British university, the authors conducted a survey among students coming toward the end of their master’s degrees and asked what their reaction would be if certain changes were introduced that would make their degrees easier to obtain. These changes included a reduction in failure rates, a reduction in drop-out rates, and a reduction in the sacrifices made in obtaining the degree in question.
A lower percentage of failures and fewer sacrifices were clearly seen as reducing the value of the degree. Lower drop-out rates did, on balance, suggest the same conclusion, but with some ambiguity. The results showed that failure and drop-out rates would need to reach very high levels before students would stop recommending their degree to others.
The conclusion was that positioning a degree successfully as a high-value, high-cost product that is difficult to obtain would produce a situation where the business interests of a university would lie in combatting grade inflation. The business and academic interests of the university in maintaining standards would tend to coincide.
Footnotes
Appendix
Authors’ Note
A subset of the data from the survey described in this article, namely those records relating to students of computing, was previously the subject of a short conference paper (Sharon & Kingsley, 2012).
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) disclosed receipt of the following financial support for the research and/or authorship of this article: This research was conducted with the help of a grant from Laureate Online Education. It had no role in the conduct of the research.