AdaBoost Regressor

Algorithm overview

AdaBoostRegressor combines weak regressors sequentially to improve predictive accuracy on non-linear regression patterns.

This algorithm is especially useful when:

• Single linear models underfit important non-linear behavior.
• You need stronger regression quality with manageable model complexity.
• Your deployment environment requires fast JS inference.

JavaScript implementation

@kanaries/ml provides AdaBoost regression in JavaScript for teams that want a stronger non-linear regressor than a single tree or plain linear model, while still keeping inference straightforward. This is useful in product analytics, pricing, or forecasting-style features implemented in Node.js or TypeScript services.

Because the estimator is available in JS, boosted regression can live in the same application layer that owns data fetching, feature building, and API responses.

Quick start

from sklearn.ensemble import AdaBoostRegressor

X = [[0], [1], [2], [3], [4]]
y = [1.0, 1.8, 3.1, 3.9, 5.2]

reg = AdaBoostRegressor(n_estimators=50, learning_rate=0.5, random_state=0)
reg.fit(X, y)
pred = reg.predict([[1.5], [3.5]])

import { Ensemble } from '@kanaries/ml';

const X = [[0], [1], [2], [3], [4]];
const y = [1.0, 1.8, 3.1, 3.9, 5.2];

const reg = new Ensemble.AdaBoostRegressor({ n_estimators: 50, learning_rate: 0.5 });
reg.fit(X, y);
const pred = reg.predict([[1.5], [3.5]]);

Quick JavaScript example

import { Ensemble } from '@kanaries/ml';

const X = [[0], [1], [2], [3], [4]];
const y = [1.0, 1.8, 3.1, 3.9, 5.2];

const reg = new Ensemble.AdaBoostRegressor({ n_estimators: 50, learning_rate: 0.5 });
reg.fit(X, y);
const pred = reg.predict([[1.5], [3.5]]);

Detailed API reference

constructor(props?: { estimator?: DecisionTreeRegressor; n_estimators?: number; learning_rate?: number; randomState?: number })

Both n_estimators/nEstimators, learning_rate/learningRate, and randomState/random_state casings are accepted.

Parameters

Algorithm

This is AdaBoost.R2 (Drucker, 1997), matching sklearn's AdaBoostRegressor with linear loss. Each round draws a weighted sample of rows, fits a regressor, computes per-sample linear losses, and reweights samples so hard examples get more focus in later rounds; learning_rate scales the weight update. Prediction is the weighted median of the estimators' predictions (not the weighted mean), which is robust to outlier estimators. A perfectly-fitting estimator is kept and stops boosting, and the ensemble is never left empty (the first estimator is retained even if its weighted error is too high), so the model does not silently collapse to predicting 0.

Methods

• fit(trainX: number[][], trainY: number[]): void
• predict(testX: number[][]): number[]

Implementation workflow

1. Train with a conservative learning rate and enough estimators.
2. Validate MAE/RMSE on holdout data and monitor overfitting.
3. Tune estimator count and learning rate for stability vs accuracy.

JavaScript deployment notes

• Compare AdaBoost regression against a simple tree or linear baseline to confirm the extra complexity is paying off.
• Tune estimator count and learning rate together rather than adjusting them in isolation.
• This model is best suited to tabular regression problems where moderate boosting depth improves fit quality.