Abstract
Dear Editor,
We read with great interest the recently published article by Terpeluk, Rogen, and Gilliam (2016) on pre-employment strength testing for nursing staff members as some of us participated in this screening examination prior to employment at the participating health system. The intent of the article, to learn the value of pre-employment strength assessment, is a worthy topic; however, we identified methodological flaws that create internal threats to the validity of the findings. We felt that we should share our concerns about methodological flaws and statistical limitations.
The authors stated that this was a “quasi-experimental, non-randomized study.” Quasi-experimental implies that two of three elements present in experimental studies (control, manipulation, and randomization) are present and that only one element is missing, typically randomization. Authors are correct stating that their study design involved a non-randomized sample, but the design also lacked control, which is established prospectively with inclusion and exclusion criteria, informed consent, control of factors that are not part of the intervention, and manipulation of the intervention. For those reasons, this study would be more appropriately called a retrospective, comparative (non-experimental) design, as comparisons were made between samples before and after implementation of a required pre-employment procedure using data retrieved retrospectively.
Authors stated that the total population was tested in the post-implementation period using the following wording: “no nurses refused to participate in the physical capability screening.” Although the statement is technically correct, it is because participation in testing was a mandatory condition of employment. Unfortunately, authors did not include cases of individuals who were hired but did not pass the examination and, therefore, were not employed. Because they were not employed, outcome data were not available for this group.
Our most significant concern was in trusting the findings presented. Authors failed to describe and discuss differences between the two groups, which is vital to understanding whether physical strength was truly related to health care benefit usage. One or more confounding factors (participant demographics or work characteristics) may have influenced the use of pharmacy and health care benefits. For example, participant age, sex, and pre-existing co-morbid conditions, specialty of the hiring unit, previous physical condition, and other unstudied factors could have been modifying variables. The slow recovery from the economic downturn that began in 2007 led to more experienced, older nurses returning to the workforce in the pre-implementation period. Nursing personnel included in the post-implementation group may have had different characteristics from the first group. They may have been younger and less likely to require pharmacy and health care services. A more detailed discussion of potential differences between groups should have been included in the limitation section beyond just stating temporal bias. At minimum, authors should have included a table that compared participant characteristics based on data that would have been available to them. Ideally, authors should have controlled for significant differences between groups using multivariate regression models to truly understand the value of strength testing as part of pre-employment screening for nursing staff members.
We appreciate authors’ concern for the health and well-being of nursing staff members, but their failure to analyze confounding variables that may have dramatically altered study results created a threat to the internal validity of study findings. We appreciate this opportunity to express our concerns.