Pre-Publication News Popularity Prediction Using Random Forest and XGBoost with SHAP: Evidence from a Regional Indonesian Portal

Abstract

Newsrooms need to judge an article's readership before they publish it. Evidence for this prepublication setting is scarce for regional Indonesian media. This study tests how far prepublication metadata predicts the relative seven-day readership of articles on kaldera.id, a
North Sumatra news portal. The hypotheses and analysis rules were pre-registered. The data
join 5,761 WordPress articles (May 2023 to October 2026) with Google Analytics 4
pageviews. The target is the tertile of seven-day pageviews within each publication month.
Thirteen pre-publication features describe timing, placement, title, body, and author source.
Logistic regression, Random Forest, and XGBoost were tuned with time-ordered crossvalidation and tested once on the most recent 1,190 articles. Random Forest reached a macro
F1 of 0.379 (95% CI 0.350 to 0.406) against 0.334 for a stratified random baseline and 0.343
for logistic regression, meeting the pre-registered criteria by a narrow margin. SHAP and
permutation importance shared only three of their top five features, so the second hypothesis
was rejected. In two pre-declared robustness checks, the advantage of the tree models over
logistic regression was no longer distinguishable from zero. Body word count was the only
feature that ranked first in every analysis. Pre-publication metadata therefore carries only a
weak signal, and editors should treat the patterns as hypotheses to test, not as rules.