What a Random Forest classifier does
A Random Forest is an ensemble of decision trees. Instead of trusting one hierarchy fitted to one sample, it trains many related but deliberately different trees and combines their predictions. Each tree receives a bootstrap sample drawn with replacement from the training rows. At each split it can consider only a random subset of features. The final class is the forest’s majority vote.
This design targets a weakness of decision trees: variance. Small changes in data can change early splits and produce a different tree. Averaging many imperfect, partly independent learners preserves nonlinear modeling while stabilizing the result. Random forests are strong baselines for tabular classification, feature screening, risk models, quality prediction, and datasets containing thresholds and interactions.
Use the Random Forest visualization
The left plot is predicted by Ensemble.RandomForestClassifier from @kanaries/ml. The right plot uses one Tree.DecisionTreeClassifier with the same maximum depth and training split. Change the number of trees, depth, dataset, or noise and compare boundaries and holdout accuracy directly.
Boundary disagreement measures how often the two fitted models assign different classes across the grid. On noisy moons or XOR data, a single tree can create brittle rectangular pockets. The forest vote often removes isolated pockets while retaining a nonlinear outline. Every grid prediction runs locally in JavaScript, so the page is also an end-to-end browser test of the public API.
Bootstrap aggregation reduces variance
A bootstrap dataset has the same row count as the training set but includes duplicates and leaves some rows out. Each tree therefore sees a different empirical problem. If individual errors are not perfectly correlated, voting cancels part of their variation. This is bagging: bootstrap aggregating.
Bagging is most effective for unstable learners such as trees. It does not automatically reduce systematic bias. A forest of shallow trees can still underfit a complex relationship, and a forest trained with leakage will repeat that shortcut very reliably. Aggregation improves an estimator; it cannot repair the definition of the learning problem.
Why random feature subsets matter
If one feature is overwhelmingly predictive, ordinary bagged trees may all choose it near the root and remain highly correlated. Restricting candidate features makes some trees discover alternative signals. An individual tree may become slightly weaker, but the collection becomes more diverse, and diversity is what lets averaging reduce variance.
maxFeatures: 'sqrt' is a common classification default. Larger subsets strengthen each split but correlate trees; smaller subsets increase diversity but may hide useful predictors too often. Treat feature sampling, depth, minimum samples, and class weighting as validation choices.
Tree count, depth, and computation
Adding trees generally makes predictions converge rather than overfit suddenly, but training time, memory, and inference cost grow. Increase the count until validation metrics and repeated-seed predictions stabilize. The exact number depends on data size, feature count, latency, and how uncertain the operational decision can be.
Depth controls the bias and variance of each member. Deep trees can capture interactions and narrow regions; bootstrap and feature randomness then smooth their aggregate. Shallow members are faster and easier to constrain but may share the same underfitting. Monitor class-specific metrics instead of optimizing only overall accuracy.
Random Forest versus one decision tree
A single tree offers a compact global rule diagram and straightforward prediction paths. A forest usually predicts better and changes less when samples move, but hundreds of paths are not a simple explanation. Permutation importance, partial dependence, accumulated local effects, and local attribution can summarize behavior, each with assumptions and failure modes.
Use a tree when a small auditable rule set is central. Use a forest when predictive stability matters more than a single hierarchy. The side-by-side chart makes this tradeoff concrete without claiming that a smooth-looking boundary is necessarily correct.
JavaScript and scikit-learn workflow
The code tabs align estimator concepts across environments: tree count, depth, feature sampling, bootstrapping, and seed. Exact trees can differ because random number generators and tie rules differ, but both workflows fit a matrix and labels, then predict new rows. Compare fixed-dataset metrics rather than serialized tree identity.
Reproducibility requires more than a constructor seed. Preserve the data snapshot, row order, feature schema, package version, preprocessing state, and validation split. If training happens in a UI, move heavier forests to a worker so rendering stays responsive; in Node.js, benchmark concurrent scoring under realistic traffic. Serialize only through a supported model format or keep deterministic training inputs, because private tree fields are not a stable interchange contract.
Split data before tuning, keep preprocessing inside cross-validation, and preserve time or group boundaries. Evaluate calibration and minority-class recall when votes drive risk decisions. Read the Random Forest JavaScript API, inspect a member in the Decision Tree visualization, or learn how randomized partitions isolate outliers in the Isolation Forest guide.