Evaluation of a smartphone-based colorimetric method for urinalysis dipstick readings in cats

Abstract

Objectives

The aim of the study was to compare the diagnostic performances of a smartphone-based colorimetric method (SBCM) for urinalysis with a semi-automated point-of-care (POC) analyser using standardised solutions and cat urine.

Methods

Artificial solutions (negative and positive quality controls, and purposely designed artificial urine) and natural urine from 216 cats were used. Two urine reagent strips were simultaneously dipped in each sample. One dipstick was read by the SBCM and the other by the POC analyser at the same time. Results for pH, proteins, bilirubin, ‘blood’, glucose and ketones were considered. Overall agreement and sensitivity, specificity and accuracy of the SBCM were determined based on selected cut-offs.

Results

For the artificial solutions, 80 comparisons were obtained for each analyte and each expected concentration. The overall agreement (exactly the same result) between the two methods was 78.4%. SBCM sensitivity, specificity and accuracy were 99.0%, 100% and 99.3%, respectively. The correlation between the two methods was almost perfect (Cohen’s kappa coefficient = 0.9851). For natural urine samples, the overall agreement (including pH) was 68.6%. Using optimal cut-offs for the SBCM determined from the results of analysis of artificial solutions, the sensitivity, specificity and accuracy of the SBCM were 100%, 76.02% and 80.5%, respectively. In this situation, the correlation between the two methods was moderate (Cohen’s kappa coefficient = 0.5401). This was mostly due to a high rate of false-positive results for bilirubin (61.1%).

Conclusions and relevance

With proper cut-off use (ie, considering positive or negative results) the SBCM evaluated here has a perfect sensitivity and appropriate diagnostic performances for proteins, ‘blood’, glucose and ketones. Based on these experimental data, this method appears suitable for dipstick urinalysis but positive results for bilirubin and proteins have to be confirmed.

Introduction

Urinalysis is a valuable and inexpensive tool for the screening and follow-up of medical status in cats. Urine dipstick analysis allows the assessment of several biochemical characteristics of urine. It also has the advantage of being easily performed by non-healthcare professionals. As urine collection is increasingly used by cat owners, the home assessment of urine dipsticks through a smartphone app could represent an interesting tool for disease detection and monitoring.

Smartphone-based urinalysis is becoming increasingly popular in human medicine^1–3 and has also been evaluated in dogs recently. ⁴ In the latter study, the kit that was used included a smartphone app but was judged inaccurate and was not recommended for clinical decisions. The main reason for this was the high rate of false-positive and false-negative results. ⁴

However, considering that visual reading is less reliable than automated analysis,^5–7 dedicated smartphone-based colorimetric methods (SBCMs) could improve the quality and transmission of urinalysis results to the attending veterinarian and hence the level of care and follow-up of small animal patients.

The aim of this study was to compare the results obtained by an SBCM with those obtained by semi-automated point-of-care (POC) urinalysis as a reference method: (1) using artificial solutions with fixed biochemical characteristics, and (2) using urine collected from cats. We hypothesised that the SBCM would provide sufficient diagnostic accuracy and good correlation with the reference method, allowing a reliable at-home usage.

Materials and methods

Samples and urine dipsticks

This study was conducted in two distinct parts. Part 1 was the validation phase with artificial solutions (quality controls and artificial urine). If the SBCM was found to be inaccurate with controlled and expected results, part 2 (validation for natural cat urine) would not have been pursued.

The negative (UQ-14 Urinalysis Control – level I; URIT Medical Electronic) and positive (UQ-14 Urinalysis Control – level II; URIT Medical Electronic) control solutions manufactured for the POC analyser were used. Expected results for the two solutions, provided by the manufacturer, are detailed in Table S1 in the supplementary material.

In order to obtain concentrations of analytes corresponding to cut-offs of interest (ie, clinically relevant cut-offs from which the result is considered positive, see the ‘Data and statistical analysis’ section), artificial urine was made from a base solution. Then, protein (Bovine serum albumin ⩾96%; Sigma-Aldrich), bilirubin (Bilirubin ⩾98%; Sigma-Aldrich), haemoglobin (Haemoglobin human; Sigma-Aldrich), glucose (Glucose 30%; B Braun) and ketones (Methyl acetoacetate 99%; Sigma-Aldrich) were added individually until the desired concentration was obtained (see Table S2 in the supplementary material).

In part 2 of the study, excess urine from routine diagnostic urine samples of cats presented to the National Veterinary School of Toulouse between October 2019 and October 2021 was used. The criterion for inclusion was residual urine volume ⩾1 ml. Samples were included regardless of the patient signalment, method of urine collection, reason for hospital admission and current medications.

Urine dipsticks manufactured for the POC analyser (14G urine reagent strips; URIT Medical Electronic) were used according to the manufacturer’s instructions.

Smartphone-based colorimetric method

The same smartphone (Redmi Note 5; Xiaomi) was used for the whole study. It was placed on a purposely designed structure, similar to previously described devices,^8,9 holding the smartphone 20 cm in front of the dipstick and including an LED lighting source (see Figure S1 in the supplementary material). The dipstick was placed in the middle of a colour chart corresponding to every possible colour for every reagent pad (see Figure S2 in the supplementary material). The SBCM app (Caremitou; Nov&Sat SAS) used in this study automatically detected the dipstick position thanks to the recognition of the red dots at each corner of the chart (Hough circle transform) described elsewhere. ¹⁰ Then, the app performed a comparison of the colour of each reagent pad to those of the chart and outputted a number corresponding to the closest colour, thus giving semi-quantitative results directly comparable to those of the POC analyser.

Semi-automated point-of-care urinary dipstick reader

The same POC analyser (URIT-31 VET; URIT Medical Electronic) was used for all tests according to the manufacturer’s instructions. It was developed for veterinary medicine and is compatible with both quality control solutions and urine dipsticks. The reading of urine reagent strips is performed by dual-wavelength spectrophotometry.

Procedure for urinary dipstick testing

For artificial solutions, two dipsticks were dipped simultaneously in the tested solution for 5 s. For natural cat urine, the two dipsticks were dipped if the volume sample was large enough. If not, one to a few drops of urine were manually placed on each of the reagent pads successively.

After gently removing the excess of solution or urine with absorbent paper, one of the dipsticks was placed in the POC analyser and the other one was placed in front of the smartphone. After being manually activated, the POC analyser read the dipstick automatically after approximatively 60 s. At the same time, a picture of the dipstick was taken through the camera of the smartphone with the dedicated app. Results for each test were reported in contingency tables for comparison.

In order to minimise variability, all the tests were performed by the same authors (VL and CG) using the same equipment.

Data and statistical analysis

Only the results for those reagent pads considered reliable for urinalysis in cats (pH, proteins, bilirubin, ‘blood’, glucose and ketones) were evaluated. ¹¹ Thus, although the dipstick provides semi-quantitative results for 14 different biochemical properties of the urine, most were ignored because they were considered unreliable (leukocytes, nitrite, urobilinogen, specific gravity) or not extensively validated for cats (ascorbic acid, microalbumin, calcium, creatinine).

Categorical results were expressed as percentages. For continuous variables, the Shapiro–Wilk Royston test was used to assess for normality. As age was the sole evaluated variable and because it was not normally distributed (P <0.001), it was reported as median (range).

The results obtained from the SBCM and the POC analyser were compared for each sample and each reagent pad. Overall agreement was calculated as a percentage of readings yielding the exact same result with both methods. An agreement percentage was also determined for each studied analyte.

Sensitivity, specificity, accuracy and Cohen’s kappa were calculated for protein, bilirubin, ‘blood’, glucose and ketone results obtained from the SBCM, using the POC analyser as reference. As urinalysis dipstick results are semi-quantitative and there are no known published clinical decision limits for veterinary urinalysis results, ⁴ cut-offs for the reference method were defined individually by analyte based on the authors’ opinion of clinically significant difference (arbitrarily defined cut-offs; Table 1). First, diagnostic performances of the SBCM were evaluated for artificial solutions (part 1). If they were judged adequate, evaluation of the SBCM using natural urine from cats could be pursued (part 2). Accuracy was calculated for each possible semi-quantitative result of the SBCM in order to determine the optimal result for each analyte (ie, the cut-off with the higher diagnostic accuracy: optimal cut-offs for the SBCM). In part 2, those cut-offs were used to calculate the sensitivity, specificity, accuracy and Cohen’s kappa for the SBCM using natural cat urine.

Table 1

Arbitrarily defined cut-offs for the reference method (semi-automated point-of-care analyser) with corresponding interpretation

Analyte	Cut-off	Negative results	Positive results
Proteins	2+	0, +/–, 1+	2+, 3+
‘Blood’	1+	0, +/–	1+, 2+, 3+
Bilirubin	1+	0	1+, 2+, 3+
Glucose	+/–	0	+/–, 1+, 2+, 3+, 4+
Ketones	+/–	0	+/–, 1+, 2+, 3+

Results for pH were compared for natural urine only, using a paired t-test with the significance set at P <0.05. As the increment scales for the POC and the SBCM were different (0.5 and 1, respectively), the results obtained from the POC were rounded to the lower integer for comparison.

All analyses were calculated with commercial software (Excel; Microsoft).

Results

Part 1: control solutions and artificial urine

A total of 240 comparisons were obtained for each analyte (80 per tested solution). The overall agreement between the SBCM and POC for control solutions and artificial urine was 78.4%. The best agreement was obtained for bilirubin (100%) and the worst agreement was for ‘blood’ (58.3%). For arbitrarily defined cut-offs applied to the SBCM and the POC, the sensitivity, specificity and accuracy were 99.0%, 100% and 99.3%, respectively (Table 2). Cohen’s kappa coefficient indicated almost perfect agreement between the two methods using artificial solutions (kappa = 0.9851).

Table 2

Sensitivity, specificity, accuracy and agreement between the smartphone-based colorimetric method and the point-of-care analyser for artificial solutions (using arbitrarily defined cut-offs)

Analyte	True positive	True negative	False positive	False negative	Sensitivity (%)	Specificity (%)	Accuracy (%)	Cohen’s kappa
Proteins	158	80	0	2	98.75	100	99.17	0.9814
‘Blood’	158	80	0	2	98.75	100	99.17	0.9814
Bilirubin	160	80	0	0	100	100	100	1
Glucose	158	79	0	3	98.14	100	98.75	0.9720
Ketones	159	80	0	1	99.38	100	99.58	0.9910
Total	793	399	0	8	99.0	100	99.33	0.9851

Therefore, optimal cut-offs for SBCM were calculated to be 1+ for proteins and bilirubin, and +/– for ‘blood’, ketones and glucose. Detailed sensitivities, specificities and accuracies for each SBCM result are provided in Table S3 in the supplementary material.

Part 2: feline natural urine

A total of 216 urine samples were available from 169 cats. One cat had six samples included, two cats had four samples, 10 cats had three samples, 16 cats had two samples and the remaining 140 cats had only one sample included. Demographic data were not available for four cats. A total of 72 female cats (one entire) and 93 male cats (five entire) were included. The median age at first inclusion was 9.1 years (range = 0.7–19.8).

The overall agreement (including pH) between the SBCM and POC for feline natural urine was 68.6%. The best agreement was obtained for ketones (92.6%) and the worst agreement was for bilirubin (36.6%). The reason for bad agreement between SBCM and POC for the bilirubin is a high rate of false-positive results (n = 132).

For cut-offs established in part 1 (ie, optimal cut-offs for the SBCM), the sensitivity, specificity and accuracy of the SBCM were 100.00%, 76.02% and 80.46%, respectively (Table 3). Cohen’s kappa coefficient indicated moderate agreement between the two methods using artificial solutions (kappa = 0.5401). The percentage of cases where clinical interpretation was identical or different between the two methods is illustrated in Figure 1.

Table 3

Sensitivity, specificity, accuracy and agreement between the smartphone-based colorimetric method (SBCM) and the point-of-care analyser for feline natural urine (using optimal cut-offs for the SBCM defined in part 1)

Analyte	True positive	True negative	False positive	False negative	Sensitivity (%)	Specificity (%)	Accuracy (%)	Cohen’s kappa
Proteins	84	89	43	0	100	67.42	80.09	0.6168
‘Blood’	73	125	18	0	100	87.41	91.67	0.8244
Bilirubin	16	68	132	0	100	34.00	38.89	0.0710
Glucose	26	187	3	0	100	98.42	98.61	0.9375
Ketones	1	200	15	0	100	93.02	93.06	0.1100
Total	200	669	211	0	100	76.02	80.46	0.5401

Figure 1

Percentages of identical and different clinical interpretations for each analyte from the results obtained with the smartphone-based colorimetric method and the point-of-care analyser from 216 cat urine samples

If the results for bilirubin were removed from the analysis, the sensitivity, specificity and accuracy of the SBCM would have been 100.00%, 88.38% and 90.86%, respectively. Cohen’s kappa coefficient would then indicate substantial agreement (kappa = 0.7642).

Without considering the pH, 23/216 (10.65%) dipsticks gave the exact same result for the five pads of interest with the two methods. However, clinical interpretation would be similar in 60/216 (27.78%) cases.

If neither pH nor bilirubin are considered (high rate of false positives), 60/216 (27.8%) dipsticks gave the exact same results with the two methods. However, clinical decisions based on the dipstick results would be similar in 129/216 (59.7%) cases. When the interpretation was not the same, the difference concerned one pad only in 72/87 (82.8%) cases.

The results obtained for the pH were significantly different between the two methods (P <0.0001). There was no difference in 66.7% of samples; 1.8% of samples had pH values lower and 31.5% had pH values higher with the SBCM compared with the POC. All different values varied by 1 unit only (Figure 2).

Figure 2

Bubble chart comparing pH results obtained by the smartphone-based colorimetric method (SBCM) and the point-of-care (POC) analyser in 216 cat urine samples. Circle sizes are proportional to the number of results (reported within or near corresponding circles)

Discussion

The SBCM evaluated in this study showed acceptable overall agreement with the chosen gold standard, a semi-automated POC analyser. To the authors’ knowledge, this is the first study validating an at-home kit for urinalysis in cats. Diagnostic performances and agreement were better with artificial solutions, which emphasises the need to validate diagnostic tools in conditions as close as possible to daily practice. However, the sensitivity, specificity and accuracy of the SBCM (100.00%, 76.02% and 80.46%, respectively) remained good using natural urine collected from feline patients. Except for bilirubin, clinical interpretation would be similar between the two methods in ⩾80% of cases for other individual reagent pads. The SBCM could thus represent a very useful tool for early detection or monitoring of some feline diseases (eg, cystitis, diabetes mellitus or even ketoacidosis). This kind of tool may encourage veterinarians to include urinalysis in their screening panels more often, especially for senior patients, as a recent survey suggests that this test is often not performed by first-opinion practitioners. ¹²

Compared with the POC analyser, the SBCM showed a high rate of false-positive results for bilirubin when evaluating natural urine despite showing a perfect agreement using artificial solutions. The main explanation for this discrepancy is that the control solutions were colourless, whereas the colour of natural urine ranged, depending on the sample, from pale yellow to amber. The risk for a false-positive result seemed to be correlated with urine specific gravity measured with a refractometer (data not shown), which is, itself, fairly correlated with urine colour, as was recently established in dogs, ¹³ and in cats with urethral obstruction. ¹⁴ With the SBCM, the slightest colour deviation from blank on the bilirubin reagent pad was considered positive. This could be easily corrected in the smartphone app algorithm by taking into account the colour of the feline natural urine. In any case, the most important criterion in early detection for any diagnostic tool is a high sensitivity rather than a high specificity, and the high rate of false-positive results does not alter this. However, any positive result obtained from the SBCM for bilirubin should be confirmed by another method.

Conversely, the sensitivity for pathological proteinuria was low with the arbitrarily established cut-off (2+). This was explained by a frequent underestimation of the colour intensity of the reagent pad by the SBCM. By using the cut-off of 1+ instead (ie, the optimal cut-off for the SBCM as established in part 1 of this study), the sensitivity increased to 100% and specificity decreased to 67.4%. This cut-off then seems more appropriate for early detection but again, any positive result (1+ and higher) should prompt confirmation with an alternative method.

The present study has some limitations. First, all tests were carried out with only one smartphone and in an experimental setting (a constant lighting environment and distance between the smartphone and the dipstick thanks to a purposely designed structure). This device is not mandatory for dipstick readings with the app but was used here to allow repeatability of the measurements. Thus, it cannot be stated that the same results would be obtained in a home environment (ie, without the purposely designed structure, with different operators and smartphones). Indeed, practical usability of a colorimetric detection app requires the app to give consistent predictions independently of changes in the illumination intensity or colour, device type and camera setting. ¹⁰ Different angles and shadowing may be alleviated by the Hough Circle Transform. ¹⁰ However, the results obtained with different cameras ¹⁰ or different light sources ¹⁵ can vary. Even though the SBCM evaluated here possesses a correcting ability depending on lighting, our study cannot fully confirm the suitability of this method in other settings.

Another limitation is the choice of the gold standard being a POC analyser rather than laboratory reference methods. ⁴ For example, urine dipsticks have been shown to be inaccurate in evaluating urine protein^16,17 and glucose ¹⁸ concentrations in cats. However, the aim of this study was to compare the results between two methods and not to establish the superiority of the SBCM compared with the POC analyser. Finally, specific disease conditions for individual cats were not taken into account and some patients were included several times, which could have impacted the results. Once again, because the goal of the study was to compare the raw results obtained from the SBCM with the chosen gold standard, performance of the app in the early diagnosis of specific conditions was beyond the scope of this study.

Conclusions

The SBCM evaluated in this study is a valuable tool for disease screening and monitoring in cats, based on its perfect sensitivity and appropriate diagnostic performance for the detection of proteins, ‘blood’, glucose and ketones in the urine. However, due to the high false-positive rate for bilirubin and proteins, positive results for these analytes should be interpreted cautiously and confirmed with another testing method.

Supplemental Material

Table S1

Expected results obtained with the semi-automated point-of-care analyser for negative and positive control solutions

Supplemental Material

Table S2

Composition of the purposely designed artificial urine

Supplemental Material

Table S3

Sensitivity, specificity and accuracy for different smartphone-based colorimetric method results (using ‘arbitrarily defined’ cut-offs for the point-of-care analyser)

Supplemental Material

Figure S1

Picture and schematic representation of the purposely designed structure for smartphone-based colorimetric method results acquisition

Supplemental Material

Figure S2

Colour chart used for the smartphone-based colorimetric method results acquisition