Group-Aware Cross-Validation

Algorithm overview

Random sample splits leak information when multiple rows belong to the same user, patient, session, or device. Group-aware splitters keep each group entirely on one side of a fold; stratified group folds additionally balance class proportions.

JavaScript implementation

@kanaries/ml exposes both splitters through utils.ModelSelection, so browser experiments and Node.js evaluation jobs can use leakage-resistant indices within the same JS workflow.

Quick start example

import { utils } from '@kanaries/ml';

const X = [[0], [1], [2], [3], [4], [5]];
const y = [0, 0, 1, 1, 0, 1];
const patientIds = ['a', 'a', 'b', 'b', 'c', 'c'];
const folds = new utils.ModelSelection.StratifiedGroupKFold({
  nSplits: 3, shuffle: true, randomState: 42,
}).split(X, y, patientIds);
console.log(folds);

Detailed API reference

new utils.ModelSelection.GroupShuffleSplit({
  nSplits?: number; testSize?: number; trainSize?: number; randomState?: number;
})

new utils.ModelSelection.StratifiedGroupKFold({
  nSplits?: number; shuffle?: boolean; randomState?: number;
})

Both expose split(X, y?, groups): FoldIndices[]. GroupShuffleSplit samples unique groups and interprets fractional sizes against the group count, not the row count. StratifiedGroupKFold requires y and greedily balances per-class proportions while never splitting a group.