Abstract
In medicine ability to communicate requires training and continuous development.
Aim
To validate the Communication Skills Ability Scale in Romanian. Study design: cross-sectional observational interrupted large case series. Sample and setting: all year 5 medical undergraduate students enrolled at UMF Carol Davila, Bucharest (UMFCD). A two field tests study: 1) 2nd semester of the 2017/2018 academic year (n = 361); 2) 2018/2019 academic year (n = 703).
Methodology
The Romanian translation of CSAS® was used.
Results
The CSAS-RO confirms the 2-Factor scale; internal consistency: Cronbach-α coefficient was 0.894 for the PAS (0.870 CSAS®) and 0.754 for the NAS (0.805 CSAS®) All item-total and item-rest correlations satisfied the criterion of more than 0.30, ranging from 0.32 to 0.71 with the exception of items 17 (field test 1) and item 11 (field test 2)
Conclusion
CSAS-RO is valid to use with medical students. The reuse of the scale with a longitudinal study design will allow to assess any new educational needs for communication ability in medical students plus add the remaining property to test (the test-retest reliability).
Introduction
Communication is an important skill in medical practice. Ability to communicate requires training and continuous development within any healthcare organisation where specific communication ethical principles apply . 1
Medical student attitudes towards communication skills are important for curriculum planners and teaching staff. In order to assess the need for training in this topic one must first evaluate the initial levels of ability to communicate. This is then followed by the assessment of their willingness to acquire and/or improve these skills as well as the means to carry out such processes, 2 In the foundation and career years, doctors should be able to demonstrate increasing competence in how to identify, acknowledge and deal with ethical, legal and professional issues on which good medical practice is based. This can be acquired through learning but also through development of communication skills.
According to proposals made by the Institute of Medical Education UK, by year 5, medical students need to “have the ability to: integrate ethical analysis of actual clinical encounters with clinical knowledge and skills and legal obligations, elaborate on common ethical arguments, propose action/decision based on this synthesis, display professional attitudes and behaviours consistent with Good Medical Practice as well as be aware of own values”. 3
A qualitative research, with five focus group discussions was conducted with 32 students, with representatives from each of the 5 years of the medical degree course at a UK university and researchers had discussions transcribed in full and the transcripts were theme analysed independently by 2 analysts. In this early-on research two assessment-related themes emerged from the analysis: namely, students’ perceptions of formative assessment and students’ perceptions of summative assessment. These assessments combine performance self-assessment and communication. While students seemed to value formative methods of assessing their communication skills, they did not appear to value summative methods like objective structured clinical examinations (OSCEs). Students had mixed views about who should assess their oral communication skills. Some students preferred self-assessment while others preferred peer assessment. Although students appeared to value medical educators assessing their communication skills, other students preferred feedback from patients. Although summative methods like OSCEs were criticised widely, students suggested that examinations were essential to motivate students’ learning of communication skills.4b
The key domains of clinical communication have been described as concentric rings (communication wheel), starting in the centre with ‘respect for others’ and moving outwards through the specific domains of communication learning. These communication domains are set within a milieu of four over-riding principles which govern not only communication, but all areas of medicine and medical practice: reflective practice, professionalism, ethics and law and evidence-based practice. 5 The over-riding principles also stay at the basis of the annual appraisal once the doctor works with a licence to practise.
Maguire and Pitceathly 6 showed key tasks in communicating with patients that good doctors should be able to perform. These are:
Eliciting (a) the patient's main problems; (b) the patient's perceptions of these; and (c) the physical, emotional, and social impact of the patient's problems on the patient and family; tailoring information to what the patient wants to know; checking his or her understanding; eliciting the patient's reactions to the information given and his or her main concerns; determining how much the patient wants to participate in decision making (when treatment options are available); discussing treatment options so that the patient understands the implications; maximising the chance that the patient will follow agreed decisions about treatment and advice about changes in lifestyle. These were particularly inspired by an expertise taken from a professor of psychiatric oncology.
They went on to say that “doctors are more likely to attend workshops or courses in communication skills if they know that substantial time will be devoted to their own agenda. Thus, they should be asked to identify the communication tasks they want help with. These will commonly include the usual tasks plus more difficult situations, such as breaking bad news, handling anger, and responding to difficult questions”. They concluded with: “doctors with good communication skills identify patients’ problems more accurately Their patients adjust better psychologically and are more satisfied with their care; doctors with good communication skills have greater job satisfaction and less work stress; effective methods of communication skills training are available, at undergraduate and postgraduate levels. The opportunity to practise key skills and receive constructive feedback of performance is essential in medical practice”.
To our knowledge no translated instrument was available to assess and evaluate communication ability and the willingness to learn this ability in medical students attending courses in Romanian Medical Schools until now. Such instrument assists with a needs based assessment of teaching and learning about an essential medical ability: that of communication skills.
Material and Methods
Objective: to validate the CSAS® in Romanian (CSAS-RO) and test its psychometric properties with a cohort of Romanian year 5 undergraduate medical students.
Study design: cross-sectional observational interrupted large case series design with all year 5 medical undergraduate students enrolled at UMF Carol Davila, Bucharest (UMFCD) during two academic years. Repeat design in two field tests: 1) second semester of the 2017/2018 academic year and 2) throughout the 2018/2019 academic year. Sensitive personal information was not required and was not collected and therefore ethical approval was not necessary considering the rules of ethics commission of the university.
Setting, sample and timeline: setting was in the largest Romanian medical school. UMFCD is a university with a medical profile quoted between positions 201–300 in the TOP 500 Shanghai (Medical Sciences category, 2020). The university education process is carried out in four faculties: Faculty of Medicine, Faculty of Dental Medicine, Faculty of Pharmacy and Faculty of Midwives and Medical Assistants. The university complies with general and sectoral EU Regulations. Medical studies are provided during a 6-year full-time programme, with a minimum of 5500 hours of theoretical activity and medical practice. Annual capacity is of 600 publicly funded students and additionally, 400 students who pay tuition fees: 250 Romanian, 150 international students. 7 A total of 361 (62% female) (Field test 1) and 703 (68% female) (Field test 2) year 5 students took part during 2nd semester (2017/2018) and throughout the 2018/2019 academic year.
The original, English version, 26-item CSAS® measures attitudes along two dimensions (Factors): the Positive Attitude Scale (PAS) and the Negative Attitude Scale (NAS). The PAS includes items 1,4,5,7,9,10,12,14,16,18,21,23,25 and the NAS includes items 2,3,6,8,11,13,15,17,19,20,22,24,26.4a A forward-backward translation was carried out in January 2018. All n = 852 students were invited, 67 were excluded (away on ERASMUS). Responders chose answers for items along a five-point Likert scale which ranges from 1 to 5 1 = strongly disagree, 2 = disagree, 3 = neither agree nor disagree, 4 = agree and 5 = strongly agree. Extreme values express the stronger attitudes. Data were collected with Google Forms, processed in Excel, analysed with IBM SPSS 23.0 and STATA 15.0.
An explorative factor analysis with direct oblimin oblique rotation was conducted and Cronbach α was calculated for both Factor scales. A descriptive indicator of the ‘badness of fit’ is provided by the Akaike Information Criterion (AIC) used with the two confirmatory factor analyses (CFA). It allows comparisons of two varying models (for example CSAS® and CSAS-RO) whereby a lower AIC belongs to the better (or preferred model) including when comparing versions in the same language (CSAS-RO versions, eg Field test 1, 2 etc). Testing used covariance matrices and the maximum likelihood method approach. We used standard psychometric tests and criteria to evaluate the acceptability, reliability, validity (internal consistency), and responsiveness of CSAS-RO (Table 1).
Psychometric tests and criteria applied (adapted from Lillevik). 15
Results
A total of 703 students out of 850 responded; 67 students were excluded due to attending an ERASMUS placement at the time of their module. An 83% response rate was recorded; mean age was 23.3 ± 0.89 years; with a 0.11 years difference (which was statistically not different) in the average age for male and female students.
Results are presented for the second pilot (Field test 2) after the re-translation of one item (item 8).
Factor loadings for CSAS-RO and CSAS®. 4
Figures 1a and 1b show the Factor loadings for CSAS-RO in the two field tests.
Screeplots oblimin oblique for factor loadings. Field test 1 (a) Field test 2 (b).
Construct validity (discriminant analysis results – gender).
For well-fitting items we expected that alpha would increase by shortening the test. Having dropped in turn items 17 and 11 Cronbach's ά coefficients increased from 0.892 to 0.894; and from 0.75 to 0.77 for NAS; results for PAS were unchanged. As a result we intend to keep all 26 items in the scale.
Discussion
All higher education medical curricula include teaching of communication ability and skills. Communication is an essential ability for any medical graduate.
The GMC UK lists in “About the Outcomes for graduates – Working with doctors Working for patients”, a minimum set of practical skills. 9
In “Outcome 2, Professional Skills – Communication and interpersonal skills” the following are highlighted:
“Newly qualified doctors must be able to communicate effectively with patients, their relatives, carers and advocates, and colleagues in a medical context, applying patient information confidentiality appropriately. They must be able to: demonstrate openness and honesty in their communication with patients, their relatives, carers or other advocates – known as the professional duty of candour; they must communicate clearly, sensitively and effectively with patients, their relatives, carers or other advocates, and colleagues from medical and other professions, by listening, sharing and responding, demonstrating effective verbal and non-verbal interpersonal skills, making adjustments to their communication approach if needed and seeking support from colleagues for assistance with communication if needed. Newly qualified doctors must communicate by spoken, written and electronic methods (including medical records) clearly, sensitively and effectively with patients, their relatives, carers or other advocates, and colleagues from the medical and other professions, in a range of situations including: where there is conflict, in difficult circumstances, such as when sharing news about a patient's condition and prognosis that may be emotionally challenging for the patient and those close to them, and when discussing sensitive issues, such as alcohol consumption, smoking, obesity or sexual behaviour, when communicating with children and young people, when communicating with people who have impaired hearing, speech or sight or learning disabilities, when English is not the patient's first language by using an interpreter, translation service or other online methods of translation, when acting as a patient advocate, when making referrals to colleagues from medical and other professions, when providing care remotely.”
The CSAS was developed early on (2002) in the development period of the UK communication curriculum, at the same time emphasis was made about communication. 6 Other researchers went later on to broaden up the approach and use qualitative methods10,13 as well as at tools to assess behavioural and social science competencies in medical education. 14
The aim here was to validate the Romanian version of CSAS. Majority translations encountered item-reducing processes as a result of specific language psychometrics evaluations but overall it proves that the all-item CSAS is a valid instrument for measuring communication skills ability with Romanian students. Its reliability coefficient is 0.892, with different results by subscales thus scoring will be done by subscales. Discriminant analysis has shown that all item scores and the scale scores are correlated with sex <0.30 as hypothesised (Table 1 and Table 3). 15
The validation has demonstrated the difficulties encountered with such studies. The general mainframe of an instrument validation follows the usual research steps where bias and/or other errors can occur. The literature cites a minimum of five types of errors which can occur during population surveys, whether epidemiological surveys or opinion polls. First one must ensure the validity of the instrument used. This is what the two fields (1 and 2) of CSAS-RO entailed. The validation for prospective use will be complete when the test-re-test reliability will also be measured. However, we can reassuringly say that we reached a valid version of the instrument which can be further used within various years of study across medical schools.
If a major project were to be carried out with the aim to develop a communication curriculum the inferential population is the entire population of medical students in the country. At this stage, a first error would be the coverage error when the researcher defines the sampling frame. This could be easily addressed given that students would be enrolled via Registry during the academic year. Once the sampling frame is ensured a second error can occur if the research does not engage with an exhaustive sample: this error is the sampling error. Fortunately, with the help of statisticians this error can be easily overcome. These two errors – the coverage and the sampling error must be addressed and minimised before the start of the survey. At this point selection bias can be identified and addressed, too.
At the implementation of the study further errors can be recorded: the participating sample can have non-response errors. These could be non-response bias, this usually happens in two situations: either when, instruments or questionnaires have not been validated and they point to ambiguous items, or when, the allocated time to fill in the survey is too short and invariably almost all respondents don’t fill in the last items of the instrument. The number of unanswered items can vary, but when half of the questions haven’t been answered this will compromise the research. The second type of non-response errors are random non-responses. These usually happen when the respondent skips an item or a few items at random, either due to non-willingness to answer or, due to not knowing what to answer and this usually happens when they don’t understand the question. This situation can also be avoided by employing the validation stage of an instrument.
Next, at the end of the filling in time respondents will form the final analysis sample and they will ensure the response rate of the survey. Whether during validation or main research, non-responses are usually analysed separately in order to distinguish between systematic versus random non-responses. In mainstream research projects non-responses can form a specific category of information.
Returning to the validation process of an instrument, if a questionnaire has not been validated or, if an instrument with low quality psychometric property criteria is used then, during measurement or mainstream application of this instrument another error occurs: the measurement error. Such errors can be entirely avoided by using only validated instruments, otherwise estimates will have low or ambiguous accuracy and precision and this can compromise the entire research by making invalid the data analysis and the obtained information from it.
Once responses are recorded and non-responses are accounted for, the first step in database management is creating a dataset which will be used for the entire data analyses plan. At this stage processing errors can occur and they must be entirely addressed before data analyses start. The starting step is data cleaning. This process follows defined principles which need to be used in order to obtain a clean dataset. The final edited dataset will then lead to good quality survey statistics based on the respondent sample.
All five types of error (coverage error, sampling error, non-response error, measurement errors and processing errors can be avoided or minimised in any project if a research protocol is followed and this usually starts with ensuring that the survey will be using a valid instrument.
CSAS-RO: an 18-Month Process
Firstly, we searched the literature to find an adequate instrument which can assist with its use to assess communication abilities as well as the willingness to learn such ability. The development of assessment of communication questionnaires includes both quantitative and qualitative (mixed) methodologies and we were aware of the tediousness of the process.
The developers of CSAS went first and validated the scale with 490 students, with year one and two medical students. This might explain some of the differences obtained in the results of the coefficients for Factor loadings (Table 2). In a second paper, with results from the same 490 students, the authors reported about the relationship between medical students’ attitudes towards communication skills learning and their demographic and education-related characteristicsThe attitudes of year one and two from two universities medical students towards communication skills learning were significantly associated with a number of demographic and education-related characteristics. Both groups of students with more positive attitudes towards communication skills learning tended to be female, tended to think their communication skills needed improving and tended not to have parents who were doctors. Both groups of students with more negative attitudes towards communication skills learning tended to think their communication skills did not need improving. The results indicated that medical students’ attitudes towards communication skills learning are associated with their demographic and education- related characteristics. These findings have a number of implications for educational practice as well as further research. 4
Furthermore, qualitative methods were used to link a further ability with communication, that of being able to personally and professionally self-assess their performance accurately. For this ability Rees et al used four semi-structured focus groups with students and professional development assessors. Assessors and students alike perceived accurate self-assessment to be difficult for students. Moreover, feedback was deemed to be crucial in helping students develop accurate self-assessment of their personal and professional behaviours. Difficulties surrounding self-assessment accuracy support the findings reported in previous literature and suggest that medical educators should encourage students to self-assess their own performance wherever possible. This can only be done if communication skills were taught and learnt. Authors go on to recommend that these results, obtained via qualitative methodology, need to be triangulated with other sources of data, such as expert panels or quantitative data, similar to information obtained including with the use of instruments like CSAS®.
Both types of ability, communication and self-assessment of own performance, can undoubtedly increase self-esteem in young graduates, leave room for development and improve quality in two processes at once: the achievement of a good educational process as well as the students’ ability to communicate as well as self-assess their performance. This approach could be employed with a third Field test of the CSAS-RO, namely, assess the willingness of learning communication abilities early on, in year one or two and re-assess the need of development of acquired skills in later years (year four or five) in order to identify remaining gaps, such as those difficulties in grasping communication techniques during all years of study and related to the value of summative methods, like for example the objective structured clinical examinations (OSCEs).
Secondly, in the case of CSAS-RO two field tests were needed. This is not unusual when the instrument has been developed in another language and needs the forward-backward translation step. Despite a reasonably sized first sample, the first Field test encountered some issues at back translation. A re-translation of one of item (item 8) was needed. That meant that a second Field test had to be deferred for the next academic year. This minor issue has considerably delayed the validation process beyond one year. The advantage however was that we did not have to engage with qualitative methodology in order to construct items from scratch. This is one major advantage when it comes to validating instruments in another language.
Thirdly, we are aware that online data collection is a much more established data collection method now compared with 15–20 years ago when this instrument was developed. A further limitation was the single distribution of the instrument which did not allow assessing the last psychometric property, the test-re-test reliability. This was due to the fact that year 5 students usually spend a limited time with our discipline. However, this stage has its limitations, most of them described as inherent of any quantitative methodology. Of the five listed errors: coverage error, sampling error, non-response error, measurement errors and processing errors, we tried avoiding all and we most likely kept them under control. However, only further re-use of the instrument will be able to assess the magnitude of such errors. Those easily controllable are the coverage and sampling error, given that students are enrolled in studies and attend classes. They participate by being fully informed about the scope of such assessments. Non-response errors were very low and can be kept low due to the short content of the questionnaire. Measurement errors relate to the valid instrument and CSAS-RO has proven to be as valid as the original scale. Processing errors relate to a set of tasks starting with data cleaning which isseemingly a tedious process in itself.. Most researchers claim that this set of tasks is key to the next step, the data analysis. The most difficult step in data cleaning was to assess the validity of online collection of responses. One advantage was the use of Google Forms, as required at each public health module. The difficulty was that we collected data via 10 modules. Hence the design of a cross-sectional interrupted case series.
Last and not least, both pilots were entirely student led via a project assignment during public health practical sessions. This accounts for a maximum 8h project management spent: from questionnaire distribution to brief report writing and a presentation of results for smaller groups. However, all the data were collated during the academic year: field test 1 during one semester (dataset 1) and field test 2 (during an entire academic year) to ensure that all students have the opportunity to participate.
The long time span for a pilot study to be carried out and the online data collection does not allow a direct comparison with other similar projects. However, we had the opportunity to distribute the original instrument CSAS® in a second parallel study where all year 5 international students participated. It was used to compare scoring results with contemporaneous samples using both English and Romanian version based on the analysis of AICs. In order to highlight the cultural adaptation of CSAS-RO we show attitudinal scores in comparison (year 5 Romanian and international students) in a second paper. Reporting was hampered by other reasons, too, for example, the COVID-19 pandemic.
Assessing the psychometric characteristics of any instrument contributes to good scientific reporting in survey statistics. Newly qualified doctors must achieve good communication and interpersonal skills at the start of their careers. CSAS-RO has proven to be a useful tool to use with medical students and is the first valid tool to use for assessing the willingness to learn communication abilities within medical schools.
Conclusions
The CSAS-RO is a valid instrument to use within medical schools: at the first Field test a 23-item scale had a Cronbach-alpha of 0.90. At the second Field test the cultural and linguistic adaptation of CSAS-RO has been optimised and all 26 items are kept in this second Romanian version with a Cronbach-alpha of 0.892. The re-translation of one item (Q8) and a good calibration of the 26-item scale gives a valid instrument to be used for assessing the communication skills ability and willingness of learning in year 5 medical students; the scale can be further used with all other Romanian medical students (years 1 to 6). The test-retest reliability which is the remaining untested psychometric property of CSAS-RO will be determined in the next Field survey.
Whether the instrument is used prospectively or simply cross-sectionally will assist in building a more structured communication training programme. The literature mentions that students who attended a communication skills course exhibited a considerable increase of communication skills in the newly developed training project where those students in an intervention group had a greater degree of self-assessed competence following training than medical students who did not participate in such training. 16 As with any training programme need assessments and evaluations are appropriate measurements of success.
Overall we recommend the use of CSAS-RO in other Romanian medical schools, public or private universities, in order to uniformly improve the quality of medical education about communication as an essential ability in practicing medicine. Despite some methodological limitations in our study, this new scale does appear to be a consistent and a stable measure of medical students’ attitudes towards communication skills learning. Further research is necessary with larger groups of students to confirm these first findings, to complete the psychometric properties with the test-retest reliability and to evaluate the utility of this scale across time and other student samples.
Footnotes
DECLARATION OF CONFLICTING INTERESTS
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this
Acknowledgements
We thank Dana Galieta Mincă for having initiated this study and the year 5 students who participated in great number as part of their public health project assignments, as well as all our colleagues at the University of Medicine and Pharmacy Carol Davila, Faculty of Medicine who helped with the data collection.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.
Ethical Approval
Not applicable, because this article does not contain any studies with human or animal subjects.
Informed Consent
Not applicable, because this article does not contain any studies with human or animal subjects.
Trial Registration
Not applicable, because this article does not contain any clinical trials.
