Using a Fine-Tuned Commercial Artificial Intelligence Model to Assess Nutrients from Photographs of Japanese Meals

Abstract

To reduce meal logging burden in diet interventions, we fine-tuned OpenAI’s GPT-4o on 1269 Japanese meal photographs (train/val/eval: 912/252/105) to estimate nutrients, using weighed food records or dietitian estimates as ground truth, and compared it with 27 non–fine-tuned models and a human dietitian. Non–fine-tuned models did poorly for fiber. Most models did well for carbohydrates, protein, and energy, while performance for salt and fat varied by model. GPT-5.1 (minimal reasoning) and non-fine-tuned GPT-4o models both provided strong accuracy, though not universally better than dietitian performance. The fine-tuned GPT-4o model’s accuracy exceeded that of the dietitian for all nutrients, with the intra-class correlation coefficient for fiber of 0.79 (95% CI 0.782-0.797) greatly exceeded the dietitian performance of 0.68, validating the accuracy of the model.