Abstract
Background:
Guidelines are discordant concerning management of patients having thrombocytopenia with cancer-associated thrombosis (CAT).
Methods:
Hospitalized adults with CAT and platelets ≤50 × 109 cells/L were managed with dalteparin 100 units/kg subcutaneously once daily. Comparator patients with CAT and platelets >50 × 109 cells/L were managed with dalteparin 200 units/kg/d.
Results:
Outcomes of 35 patients with thrombocytopenia (mean platelet count 26 ± 8.3 × 109 cells/L) and 58 comparator patients (mean platelet count 155 ± 75 × 109 cells/L) were evaluated. In all, 2 (5.7%) patients in the thrombocytopenia group and 1 patient (1.9%) in the comparator group experienced new-onset venous thromboembolism (odds ratio 3.31, 95% confidence interval [CI] 0.29-37.90, P = .556). The incidence of bleeding in patients with thrombocytopenia (8.6%) was similar to that in comparator patients (9.4%; risk ratio 0.94, 95% CI 0.37-2.39, P = .607).
Conclusion:
In hospitalized patients having thrombocytopenia with CAT, reduced-dose low-molecular-weight heparin was generally efficacious.
Introduction
An association between abnormal blood coagulation and cancer was first described in the 1860s by the French physician Armand Trousseau. Trousseau syndrome, as the condition still is known, arose from clinical observations in a series of patients with gastric carcinoma in whom a high incidence of venous thromboembolism (VTE) was reported. 1 Heightened appreciation for both the cause and the prevalence of this condition was gained in the 1930s with demonstration of plausible hematological etiologies for secondary thrombophilia and associated thrombotic complications 2 and reporting of autopsy findings from more than 4000 patients that revealed not only the presence of multiple thromboses in 15% to 30% of patients with tumors in various organs but also documentation of fatal thromboembolic disease in more than 50% of patients with pancreatic cancer. 3
More recently, epidemiological investigations have identified and at least partially defined the overall incidence, severity, and prognostic implications of cancer-associated thromboembolism (CAT). Among patients presenting with deep vein thrombosis (DVT) or pulmonary embolism (PE), the presence of a malignant neoplasm has proven to be an independent risk factor for abnormal blood clotting manifesting as symptomatic VTE. 4 An assessment of risk factors among more than 5000 adults revealed that, as compared to those without cancer, the overall risk of VTE was increased approximately 7-fold in patients with malignancy. Among patients in this series with hematological malignancies, the risk of VTE was increased 28-fold. 5 A review of Medicare claims for hospitalization among more than 1.2 million persons older than 65 years of age revealed that the probability of death was increased more than 3-fold in patients with DVT or PE and cancer as compared to those with DVT or PE and no malignancy. 6
Cancer chemotherapy poses additional thromboembolic risk and, in some patients, this can worsen prognosis. 7,8 Among 4400 patients with cancer who received outpatient chemotherapy, thromboembolism was listed as the observed cause of death in 9% of cases. 9 When compared to the general population, the risk of having a thromboembolic episode was higher in patients with malignancy treated with a cancer-directed chemotherapy regimen (relative risk 18.5; 95% confidence interval [CI] 11.9-28.7) with the greatest incidence of thromboembolism occurring within the first year after cancer diagnosis (incidence rate 15.0; 95% CI 13.8-16.2). 10 Corollary with the increased prevalence of thromboembolic events experienced by chemotherapy-treated patients is a near doubling of health care resource utilization and costs. 11 Predictors of chemotherapy-related thromboembolism include advanced age 12 and use of common antineoplastic and supportive agents such as platinum analogs, 12,13 bevacizumab, 13 thalidomide, 14 and epoetin alfa. 13,14
Although compensatory hemostatic mechanisms may be active in many patients, 15 the presence of thrombocytopenia with certain malignancies portends significant risk of excessive spontaneous or traumatic hemorrhage. 16,17 When coupled with the fact that conventional antithrombotic treatments tend to be comparatively ineffective for prevention or treatment of CAT, 18 –20 thrombocytopenia renders management of VTE in patients with cancer highly problematic. Indeed, major bleeding has been found to be not only a relatively frequent and severe complication but also an independent predictor of mortality in patients with anticoagulated cancer. 21
For these reasons, patients with CAT and incident thrombocytopenia are managed conservatively in our hospital. Hematological outcomes associated with the use of a standardized antithrombotic protocol for patients having thrombocytopenia with CAT over a 32-month period are the subject of this report.
Methods
This study was approved by our local ethics committee (Colorado Multiple Institutional Review Board, protocol No. 12-0673) and the Hospital Research Review Committee.
Data Source and Sample Selection
Patients older than 18 years of age admitted to the University of Colorado Hospital between September 1, 2009, and March 31, 2012, with a cancer diagnosis who received therapeutic anticoagulation for primary treatment or secondary prevention of VTE under auspices of the Inpatient Anticoagulation-Thrombosis Management Service were identified by review of manual and computerized inpatient prescription data (Epic Willow, Epic Systems Corporation, Verona, Wisconsin), and their medical records were retrospectively examined. Patients were included if they (1) received treatment for confirmed malignancy within the most recent 6 months; (2) had objective documentation of VTE that was either acute (index DVT or PE diagnosis made within the past 30 days) or subacute (diagnosis made within the past 1 to 3 months) or chronic (diagnosis made >3 months ago); and (3) received at least 3 doses of subcutaneous low-molecular-weight heparin (LMWH). Dalteparin in particular was selected for study on the basis of its availability within the hospital formulary as the preferred or first-line therapeutic agent for its approved indications and its comparatively high level of usage in inpatients with cancer.
Patients with cancer and acute, subacute, or chronic VTE with documentation of concurrent thrombocytopenia (defined as having at least 1 platelet count >20 but ≤50 × 109 cells/L) were managed with routine diagnosis-related hospital care and continuous antithrombotic therapy with dalteparin 100 units/kg subcutaneously once daily. Dalteparin was withheld if platelets fell below 20 × 109 cells/L, despite transfusion support or if evidence of bleeding was present. Conversely, if platelet counts recovered or increased to >50 × 109 cells/L, the dose of dalteparin was increased to 200 units/kg/d.
To determine whether observations in patients with thrombocytopenic cancer receiving dalteparin treatment differ from those occurring in dalteparin-treated patients having cancer without or with less severe thrombocytopenia, the medical records of a cohort of consecutively admitted adults with CAT and platelet counts >50 × 109 cells/L managed with standard hospital care and dalteparin therapy during an inpatient stay within the same period of observation were evaluated. These patients received subcutaneous dalteparin 200 units/kg/d. In a similar fashion, recipient patients were identified through directed searches of inpatient health records. Electronic health records were reviewed, relevant data were recorded, and findings in the nonthrombocytopenic cohort were recorded, systematically aggregated, and statistically compared to those in the thrombocytopenic group.
Patients were excluded if dalteparin therapy was administered for less than 72 hours, if they received other anticoagulant medications concomitantly or if their medical records contained incomplete, equivocal, or contradictory information.
Outcome Measures
The medical records of patients with cancer who received standard hospital care along with dalteparin treatment were examined for signs of new or recurrent thrombosis and hemorrhage. Salient demographic and clinical information was recorded. Along with patient-specific symptomatic and subjective details, evidentiary data including abnormalities in coagulation testing, imaging, and ultrasound results were sought which would satisfy diagnostic criteria for failure to attain the primary efficacy outcome of clot resolution or prevention of development of new or recurrent VTE and associated thrombotic complications. Evidence of acute and/or chronic hemorrhage, the primary safety outcome, was evaluated through assessment of trends in data regarding blood counts as well as recorded signs or symptoms of blood loss or overt bleeding, transfusion requirements, and administration of hemostatic agents.
Statistical Analysis
Results are expressed as the mean ± standard deviation. Incidence rates were compared by construction of 2 × 2 contingency tables and statistical testing with chi-square or Fisher exact probability test. Continuous variables were tested as discrete populations with Student t test. Analyses were conducted using SAS version 9.2 (SAS Institute Inc, Cary, North Carolina).
Results
During the 32-month period of observation, 35 inpatients with CAT who met thrombocytopenic inclusion criteria received subcutaneous dalteparin therapy during 42 unique admissions. In this same period, 58 comparator patients having CAT without or with less severe thrombocytopenia received dalteparin during 69 admissions.
Patient Characteristics
Overall, patients ranged in age from 21 to 85 (mean 57 ± 14) years, and 57% were male (Table 1). Patients in the thrombocytopenia group had a higher mean body weight than those in the comparator group (85 ± 18.4 vs 73 ± 17.2 kg, respectively; P = .002). The numerical prevalence of nonmalignant comorbidities tended to be greater in patients with thrombocytopenia, and immunological disorders were more common in patients with thrombocytopenia than in comparator patients (9 [26%] vs 2 [3%], respectively; P = .002). This difference was primarily due to the occurrence of immune system complications following hematopoietic stem cell transplantations (HSCTs). The HSCT was more prevalent among patients with thrombocytopenia than among comparator patients (21 [60%] vs 3 [5%], respectively; P < .001).
Clinical Characteristics of Patients With Cancer and Acute or Subacute Venous Thromboembolism.
Cancer diagnoses varied across groups (Table 1). Although no significant differences were identified in the prevalence of solid organ tumors in either group, a significantly greater proportion of patients with thrombocytopenia had hematological or lymphatic malignancies (89%) than comparator patients (17%; P < .013). Correspondingly as described earlier, more patients in the thrombocytopenia group than in the comparison group had undergone HSCT.
With several notable exceptions, documented thromboembolic disorders were similarly distributed between groups (Table 2). Acute PE was confirmed in 12 of 58 comparator patients (21%) when compared to 1 of 35 patients with thrombocytopenia (2%, P = .027). As compared with patients having thrombocytopenia, a history of previous peripheral VTE was somewhat more prevalent among comparator patients (24 [29%] of 58 vs 5 [12%] of 35; P = .044). Subacute or chronic VTE was the most commonly documented type of thromboembolic disorder. Among these thromboses, persistent blood clots associated with central venous access devices (CVADs) were the most prevalent, affecting more than half of patients in the thrombocytopenia group. Patients with thrombocytopenia were more likely to have subacute or chronic CVAD-related VTE than comparator patients (P = .002).
Thromboembolic Disorders Documented in Patients With Cancer and Acute or Subacute Venous Thromboembolism.a
Abbreviations: CVAD, central venous access device; VTE, venous thromboembolism.
a Some patients had multiple thromboembolic disorders.
Outcomes
Mean dalteparin dosages were 103 ± 23.8 units/kg/d in the thrombocytopenia group and 185 ± 40.3 units/kg/d in the comparator group (P < .001). Total durations of in-hospital dalteparin treatment were 16 ± 11.0 days in the thrombocytopenia group and 6 ± 4.7 days in the comparator group (P < .001).
Clinical findings suggestive of failure to attain the primary efficacy outcome of clot resolution or prevention of development of new or recurrent VTE were objectively confirmed in 2 patients in the thrombocytopenia group and 1 patient in the comparison group (5.7% and 1.9%, respectively; odds ratio 3.31, 95% CI 0.29-37.90; P = .556; Figure 1).
Occurrence of new or recurrent thrombosis in dalteparin-treated patients with acute or subacute venous thromboembolism.
All patients failing to attain the primary efficacy outcome were at high risk of VTE. One patient failing treatment in the thrombocytopenia group was a 68-year-old female status-post allogeneic HSCT for diffuse large B-cell lymphoma who presented with pneumonia in the setting of immunosuppression and graft versus host disease that progressed to septic shock, multiorgan dysfunction syndrome, and refractory hypotension associated with right lower lobe segmental PE, portal vein thrombosis, and renal vein thrombosis that ultimately proved fatal. The surviving patient with thrombocytopenia (nadir platelet count 9 × 109 cells/L) who failed inpatient dalteparin treatment was a 48-year-old male with nonseminomatous metastatic testicular cancer who presented with extensive bilateral DVT, despite self-administered enoxaparin therapy that was dose-adjusted for an estimated glomerular filtration rate <30 mL/min whose DVT subsequently failed to resolve on a continuous intravenous infusion of unfractionated heparin. The patient in the comparator group who failed treatment was a 53-year-old man with stage IV nonsmall-cell lung cancer metastatic to brain and mediastinum who presented with pleuritic chest pain and hypoxemia associated with an acute right anterior PE that occurred, despite adherent self-administration of full-dose dalteparin therapy for a left lower extremity DVT that was first diagnosed during the week prior to admission.
Evidence for bleeding of any severity was identified in 3 patients in the thrombocytopenia group and 5 patients in the comparison group (8.6% and 9.4%, respectively; risk ratio 0.94, 95% CI 0.37-2.39; P = .607; Figure 2). Prominent risk factors for hemorrhage were present in all patients with bleeding.
Occurrence of hemorrhage in dalteparin-treated patients with acute or subacute venous thromboembolism.
In treated patients with thrombocytopenia, risk of bleeding was exaggerated due to the presence of etiological sources of hemorrhage in addition to abnormally low platelet counts. In the 3 patients experiencing bleeding, risk factors and their associated hemorrhages, respectively, included hematemesis and hematuria in chronic peptic ulcer disease, severe hematuria with indwelling ureteral stents acutely placed for obstructive uropathy, and severe epistaxis with chronic self-administered full-dose dalteparin prophylaxis that was continued, despite an undetected drop in platelet count to 34 × 109 cells/L.
Comparator patients treated with full-dose dalteparin who had hemorrhage also had important risk factors. One patient with upper gastrointestinal (GI) hemorrhage had a long history of recurrent GI bleeds and hematuria. Another patient developed serosanguineous oozing from an exfoliative foot wound that required surgical intervention. Three treated patients developed postoperative wound site hemorrhage following extirpation or excisional surgery for thyroid, oropharyngeal, or glossal cancer.
At the commencement of inpatient antithrombotic treatment, values for most hematological indices were slightly but significantly lower in patients with thrombocytopenia. Mean initial hemoglobin concentration was 9.0 ± 1.49 g/dL in the thrombocytopenia group and 10.0 ± 1.81 g/dL in the comparator group (P = .007), and mean initial hematocrit was 28.0% ± 3.69% in the thrombocytopenia group and 32.0% ± 5.65% in the comparator group (P = .002). During inpatient dalteparin treatment, hematocrit declined in both groups by an average of 4 percentage points to 24.0% ± 3.25% and 28.0% ± 4.35%, respectively.
Patients with thrombocytopenia had greater overall transfusion requirements. In patients with thrombocytopenia, transfusion of packed red blood cells (PRBCs) was recorded in 79% of admissions, and platelet transfusions were recorded in 88% of admissions. In contrast, PRBC transfusions were recorded in 25% of admissions in the comparator patients (P = .001). Mean PRBC transfusion amounts were similar in patients with thrombocytopenia and in comparator patients (5.2 ± 9.90 and 7.8 ± 15.49 units, respectively; P = .114). As expected, platelet transfusions were required by a higher proportion patient admissions in the thrombocytopenia group (33 [88%] of 42 ) than in the comparator group (1 [2%] of 69; P < .001). Mean number of units of platelets transfused during admissions in the thrombocytopenia group (7.6 ± 15.17, range 1-31) was numerically greater than in the comparator group (3 units [1 patient]). Summary assessments of transfusion requirements indicate that overall differences between groups are more closely associated with differing cancer diagnoses and intervention- and treatment-related anemias than acute hemorrhage.
Discussion
In our study population, patients having thrombocytopenic cancer with evidence of acute, subacute, or chronic VTE were effectively managed using antithrombotic treatment with reduced-dose subcutaneous dalteparin (100 units/kg daily). Rates of clot recurrence or extension (5.7%) and clinically relevant hemorrhage (8.6%) observed in our patients with thrombocytopenia were comparable to rates documented in patients having cancer without or with less severe thrombocytopenia treated with full-dose dalteparin. These rates are also comparable to findings in patients with nonthrombocytopenic cancer enrolled in randomized controlled trials. 19,20,22
In our series, all patients who experienced recurrent VTE, despite treatment had prominent risk factors for clots. In terms of likelihood of treatment failure, a review of these patients suggests that the importance of individual differences in clot risk may outweigh the influence of most differences in clinical characteristics between treatment groups (Table 1). This suggests an opportunity for further risk stratification and individualization of antithrombotic dosing and other countermeasures.
Most contemporary treatment recommendations for management of patients with CAT are based at least in part upon findings of the landmark Randomized Comparison Low-Molecular Weight Heparin Versus Oral Anticoagulant Therapy for the Prevention of Recurrent Venous Thromboembolism in Patients with Cancer (CLOT) trial published in 2003. 22 This 6-month study affirmed that acute treatment with LMWH followed by vitamin K antagonist therapy was inferior to long-term treatment using essentially full-dose LMWH with regard to the primary efficacy outcome of prevention of recurrence of DVT, PE, or both. The LMWH regimen used in this trial (dalteparin 200 units/kg subcutaneously once daily for 1 month followed by dalteparin 150 units/kg subcutaneously once daily for 5 months) is explicitly or at least tacitly recommended for treatment and prevention of thromboembolism in most patients with cancer according to clinical practice guidelines published by the American College of Chest Physicians (ACCP), 23 the Cardiovascular Disease Educational and Research Trust, 24,25 the Groupe Francophone Thrombose et Cancer of the French Institute of Cancer, 26,27 the American Society of Clinical Oncology (ASCO), 28 the National Comprehensive Cancer Network (NCCN), 29 the European Society of Medical Oncology (ESMO), 30 the Australian National Health and Medical Research Council (NHMRC), 31 and the Scientific and Standardization Committee of the International Society on Thrombosis and Haemostasis (ISTH). 32 Accordingly, for management of CAT, LMWH treatment and extended-duration, therapeutic- or full-dose dalteparin therapy in particular may be considered the current standard of care.
Patients with CAT and thrombocytopenia (defined as a platelet count less than 75 × 109 cells/L) were excluded from the CLOT trial, 22 and practice guidelines are broadly discordant regarding means to best manage these patients during hospitalization. Some guidelines do not address issues relating to recommendations or suggestions for preferential use of antithrombotic therapy with either full therapeutic doses or dosage concession or adjustment in patients having thrombocytopenia with CAT. 23 –27 However, the ASCO, 28 NCCN, 29 ESMO, 30 and NHMRC 31 clinical practice guidelines categorize anticoagulation as contraindicated or not recommended in patients with platelet counts less than 50 × 109 cells/L (Table 3). Only the recently published ISTH guidelines 32 offer recommendations for treatment of CAT in the presence of thrombocytopenia. These recommendations, which are strongly supported by findings of the present study, are as follows:
Clinical Practice Guideline Recommendations for Antithrombotic Therapy in Patients With Cancer, Acute or Subacute Thrombosis, and Platelet Counts ≤50 × 109 cells/L.
Abbreviations: ACCP, American College of Chest Physicians; ASCO, American Society of Clinical Oncology; CDERT, Cardiovascular Disease Educational and Research Trust; ESMO, European Society for Medical Oncology; INCa, French Institute of Cancer; ISTH, International Society on Thrombosis and Haemostasis; LMWH, low-molecular-weight heparin; N/A, not addressed; NCCN, National Comprehensive Cancer Network; NHMRC, National Health and Medical Research Council (Australia).
CAT with platelet count ≥50 × 109/L: full therapeutic doses of anticoagulation without platelet transfusion;
Acute CAT and thrombocytopenia (platelet count <50 × 109 cells/L): platelet transfusion to maintain a platelet count ≥50 × 109 cells/L along with full therapeutic doses of anticoagulation; if platelet transfusion is not possible or is contraindicated, suggest insertion of a retrievable filter and removal of the filter when the platelet count recovers and anticoagulation can be resumed;
Subacute or chronic CAT and thrombocytopenia (platelet count <50 × 109 cells/L): reduce the dose of LMWH to 50% of the therapeutic dose or use a prophylactic dose of LMWH in patients with a platelet count of 25 to 50 × 109 cells/L; discontinue anticoagulation in patients with a platelet count <25 × 109 cells/L. 32
Several reviews of management of CAT have been recently published. 33 –39 Among these, specific recommendations for anticoagulant dosing in patients with thrombocytopenia are found in only 1 report. 39 These reasonable but apparently untested recommendations for individualized treatment advise that consideration should be given to withholding anticoagulation in all patients (possibly excepting those at the highest level of thrombotic risk) if severe persistent thrombocytopenia (platelets <20 × 109 cells/L) refractory to transfusion support is present. When the platelet count is between 20 and 50 ×109 cells/L, the dose of LMWH should be reduced by half, that is, dalteparin 100 units/kg subcutaneously once daily. In patients with thrombocytopenia and an acute serious thromboembolic event, platelet transfusion support to maintain the platelet count above 50 × 109 cells/L may be sufficient to allow full therapeutic anticoagulation (dalteparin 200 units/kg daily). 39 These recommendations are generally congruent with the ISTH guidelines 32 outlined earlier, and, in a similar fashion, these recommendations are likewise supported by our findings. Further study is warranted. This is currently the topic of continuing evaluations of anticoagulant therapy using automated review methods in a broadened population of patients with cancer managed across our multiple-hospital and affiliated clinic health system.
Although experience is limited with regard to antithrombotic therapy in patients having thrombocytopenia with cancer, the few reports published to date generally support the individualized treatment strategy described earlier. In the most recent of these reports, 39 patients with chemotherapy-induced thrombocytopenia (44% considered high risk for thromboembolic events) were managed with LMWH bridging around the time of repeated chemotherapy administration. 40 The LMWH therapy was begun the day following chemotherapy using enoxaparin 40 mg or nadroparin 3800 units subcutaneously twice daily in high-risk patients (history of prior arterial thromboembolism or DVT or PE) or once daily in low-risk patients (with risk factors but no prior DVT or PE). The LMWH therapy was discontinued when the platelet count was less than 30 × 109 cells/L and reinitiated 12 to 24 hours after attainment of a stable platelet count equal to or greater than 30 × 109 cells/L. Throughout a mean treatment duration of 6.8 days, a thromboembolic event occurred in 1 patient in the high-risk group (2.6%) and major bleeding occurred in 2 (5.1%) patients, both in the high-risk group. 40 In a second experience report, 41 10 patients with hematological cancer treated with intensive chemotherapy were given thromboprophylaxis or thromboembolic treatment for catheter-related central venous thrombosis with enoxaparin in dosages of 0.25 to 0.5 mg/kg subcutaneously once or twice daily during periods of severe thrombocytopenia. Duration of treatment varied from 1 to 26 days among patients with platelet counts ranging from 1 to 22 × 109 cells/L. With a single exception, all patients received platelet transfusion support. Treatment was considered successful in all patients and none developed major bleeding. 41
In an earlier experience, 5 adults aged 21 to 41 years with objectively confirmed upper extremity or abdominal VTE associated with an indwelling central venous catheter and thrombocytopenia (4 with platelet counts ≤20 × 109 cells/L) due to the conditioning regimen for autologous HSCT were treated with enoxaparin 40 mg subcutaneously twice daily for 14 days followed by enoxaparin 40 mg subcutaneously once daily for at least 2 months. 42 Signs and symptoms of venous occlusion resolved within 1 to 4 days in all patients. Repeated ultrasonography demonstrated complete thrombus resorption in 3 patients and partial recanalization in 2 patients. No bleeding or hemorrhagic complications occurred, and platelet counts returned to normal following transplantation in all patients. 42
Finally, indirect evidence from retrospective, 43 prospective, 44 and randomized trials 45 in patients with obligate thrombocytopenia following conditioning for HSCT demonstrated that administration of customary prophylactic doses (approximately half of usual therapeutic dose amounts) of LMWH was generally safe and effective in prevention of hepatic veno-occlusive disease in these patients.
Our study has several important limitations. It was performed with a retrospective, uncontrolled design and an assessment of care provided at the discretion of autonomous physicians. Our findings are reflective of experience from a single institution with a relatively small sample size. There was no institutional mandate for recording of specific clinical findings of interest. The medical records used to identify signs and symptoms of VTE or hemorrhage did not always address specific clinical criteria required for systematic evaluation of patients’ clotting status. Nonetheless, this assessment of case findings offers a clinical perspective taken from typical contemporary acute patient care and, as such, it is representative of a broad spectrum of hematology and medical oncology practice. When viewed in the context of LMWH dosage recommendations provided by others, 32,39 the generally favorable findings from our experience in anticoagulation of patients with thrombocytopenic cancer appear to be within expectations.
In conclusion, our experience suggests that hospitalized patients with CAT and platelet counts ≤50 × 109 cells/L may be effectively treated with subcutaneous dalteparin 100 units/kg daily. Full-dose dalteparin therapy (200 units/kg daily) usually may be resumed when platelet counts rise to >50 000 × 109 cells/L.
Footnotes
Declaration of Conflicting Interests
The author(s) declared no potential conflicts of interest with respect to the research, authorship, and/or publication of this article.
Funding
The author(s) received no financial support for the research, authorship, and/or publication of this article.