glmtree builds decision trees with a logistic regression at each leaf: the tree cuts the
population into segments, and each segment gets its own, interpretable, scoring function. The
segmentation is searched jointly with the regressions by a stochastic EM algorithm, as described in
Formalization and study of statistical problems in Credit Scoring, Ehrhardt A. (see the
manuscript or the
web article).
Status. The package was archived on CRAN on 2025-10-03 (a vignette needed a suggested package that was missing on one check flavor) and a re-submission is in preparation. Meanwhile, install it from GitHub.
# install.packages("pak")
# pak::pak("adimajo/glmtree")
install.packages("glmtree", repos = c('https://adimajo.r-universe.dev', 'https://cloud.r-project.org'))
library(glmtree)
set.seed(1)
data <- generateData(n = 2000, scenario = "tree") # 3 segments, 3 different regressions
features <- c("x1", "x2", "x3")
fit <- glmtree(data[, features], data$y, K = 5, iterations = 100,
ctree_controls = partykit::ctree_control(alpha = 0.1, minbucket = 100, maxdepth = 4))
fit # number of segments, criterion, the tree
predict(fit, data[1:5, features]) # probabilities
predict(fit, data[1:5, features], type = "segment") # segments
fit@best.tree$glms # the logistic regression of each segment
partykit also exports a function called glmtree() (model-based recursive partitioning). If you
attach both packages, call ours as glmtree::glmtree(). See vignette("glmtree") for the interface
(validation, test, criterion, burn_in, patience, ...).
The numbers below come from inst/benchmarks/real_data.R (20 random stratified 70/30 splits per
dataset, 100 SEM iterations, K = 5; the full table and the raw results are in
inst/benchmarks/results/). They are test-set Gini coefficients (2 AUC - 1, higher is better), means
over the splits. Tree-based methods use leaves of at least 5% of the training data and depth 4.
glmtree selects the tree with the BIC on the training data, glmtree (valid.) uses 25% of the
training data as a validation set and the Gini. MOB is partykit::glmtree(), the
model-based recursive partitioning, with all features both as regressors and as partitioning variables.
| Dataset | logit | ctree |
MOB | random forest | glmtree |
glmtree (valid.) |
|---|---|---|---|---|---|---|
| Simulated, 3 segments (n = 3000) | 0.248 | 0.568 | 0.698 | 0.657 | 0.650 | 0.644 |
| Simulated, 1 regression (n = 3000) | 0.672 | 0.619 | 0.672 | 0.606 | 0.672 | 0.670 |
| German credit (1000 × 20) | 0.552 | 0.439 | 0.436 | 0.579 | 0.514 | 0.393 |
| Pima diabetes (532 × 7) | 0.704 | 0.591 | 0.704 | 0.665 | 0.697 | 0.633 |
ISLR Default (10000 × 3) |
0.896 | 0.867 | 0.896 | 0.849 | 0.894 | 0.893 |
AER CreditCard (1319 × 10) |
0.670 | 0.608 | 0.654 | 0.692 | 0.667 | 0.668 |
ISLR Caravan (5822 × 85) |
0.439 | 0.400 | 0.439 | 0.426 | 0.413 | 0.379 |
What this says, honestly:
glmtree is not a general-purpose accuracy booster. It helps when the population really is made
of segments with different regressions: on the simulation with three segments it goes from 0.248 (one
logistic regression) to 0.650, using 4.1 segments on average (three are true). On the five real datasets
no single-tree method (ctree, MOB, glmtree) beats a plain logistic regression: glmtree is between
0.002 and 0.038 below it, because the data are close to a single regression and the chain mostly keeps
one or two segments (1.2 to 1.7 on average). Only the random forest is ahead, on German credit
(+0.027 ± 0.007, paired mean ± standard error) and CreditCard (+0.022 ± 0.006).partykit::glmtree() (MOB) is the stronger and faster alternative on these data. On the
simulation with segments MOB scores higher than glmtree (0.698 against 0.650) and takes 1.4 s per fit
against 11 s. On the real data sets MOB either does not split (it is then identical to the logit: Pima,
Default, Caravan) or loses (German credit 0.436, CreditCard 0.654), where glmtree stays close
to the logit. glmtree is neither faster nor more accurate than MOB in this benchmark; what it offers
is a different, likelihood-based search (a latent segment variable), a choice of criterion (BIC, AIC,
held-out Gini) and ctree for the segmentation.Caravan), measured with six fits running
in parallel. About 70% of the time is spent in partykit::ctree, which is refitted at every iteration.
Compared with this package's own version 0.3.1, the rewrite of version 0.4.0 made an iteration about
twice as fast for the same accuracy (2.4×, 2.2× and 2.0× for 1000, 3000 and 10000 observations, 50 iterations;
inst/benchmarks/speed.R).patience = 25 the chain stops after about 51 iterations in about 5 s at
a Gini of 0.646, against 75 s for 800 iterations: use patience for a quick answer, and a large
iterations if the extra 0.03 Gini matters (inst/benchmarks/convergence.R).Reproduce with Rscript inst/benchmarks/real_data.R 20 100 and Rscript inst/benchmarks/convergence.R
(they need partykit, ranger, scorecard, ISLR, AER; about an hour).
?glmtree, vignette("glmtree").