What k-nearest neighbors does
K-nearest neighbors, or KNN, is a supervised learning method that predicts from local examples. Training is deliberately simple: the classifier stores feature vectors and their labels. When a new sample arrives, it measures the distance to the training set, finds the closest k samples, and combines their labels in a vote. The intuition is that observations near one another are likely to share an outcome.
KNN can represent nonlinear class boundaries without learning coefficients or constructing a tree. That makes it a useful baseline for classification, a teaching model for distance-based learning, and a practical method when datasets are modest and local similarity is meaningful. The same neighborhood idea also supports regression, imputation, recommendation, and anomaly scoring.
A complete KNN visualization in JavaScript
The interactive map fits Neighbors.KNearestNeighbors from @kanaries/ml. Choose blobs, interlocking moons, or an XOR pattern; then change k, the distance metric, or the weighting rule. Every colored cell is a real browser-side prediction. There is no pre-rendered illustration and no backend Python process.
Click inside the map to add a labeled training observation. Use the class controls to decide its label. Drag the diamond-shaped query point and dashed lines identify its current neighbors. The table exposes their rank, distance, class, and effective vote weight, connecting the final color to the individual evidence behind it.
How k changes bias and variance
Set k to one and the classifier gives each training sample its own territory. This can capture fine structure, but one mislabeled or noisy sample can create an island in the decision map. Increase k and votes average over a broader area. The boundary becomes smoother and less sensitive to individual samples, but a large neighborhood may overwhelm minority classes or merge genuinely separate regions.
This is the classic bias-variance tradeoff in a visible form. A very flexible boundary has low training bias and high sensitivity; a heavily smoothed boundary makes stronger assumptions. Select k with cross-validation rather than by judging the training plot alone. In binary tasks, an odd value can reduce ties, though class imbalance and distance weighting still matter.
Distance metrics and weighted voting
Euclidean distance measures the straight-line separation between points. Manhattan distance adds horizontal and vertical differences. Change the selector on the XOR data and watch the regions respond: L2 neighborhoods are circular, while L1 neighborhoods have diamond geometry. Neither is universally superior. The metric should reflect how feature differences combine in the application.
Uniform voting gives every selected neighbor one vote. Distance voting increases the influence of close observations, which can preserve local detail even with a moderately large k. An exact match needs special handling to avoid division by zero; the library resolves this case while the table caps its displayed reciprocal at a safe numeric denominator.
Scaling, performance, and responsible interpretation
Because KNN depends on distances, features must be comparable. A yearly-income column measured in thousands will dominate a zero-to-one score unless you scale or deliberately weight them. Fit the scaler on training data only, then apply the same transformation to validation and production samples. Missing values, categorical variables, and irrelevant dimensions also need thoughtful preprocessing.
Prediction compares a query with stored observations, so basic KNN becomes slower as the dataset grows. Spatial indexes can help in low dimensions, while approximate-neighbor methods are common at large scale. High-dimensional spaces introduce another challenge: distances become less discriminative, often called the curse of dimensionality. Feature selection or a method such as PCA can help, provided validation confirms that useful class information is retained.
JavaScript KNN and scikit-learn
The code tabs show parallel APIs. In JavaScript, construct KNearestNeighbors with positional settings for k, weights, and metric, call fit, then predict. Scikit-learn expresses the same choices as named arguments. Both implement the familiar estimator workflow, making it straightforward to prototype in Python and move an interactive experience into a browser or Node.js application.
Use the KNN JavaScript API guide for method details. Compare this local, nonparametric boundary with the linear probabilities in the logistic regression calculator, or open the K-Means visualization to see what changes when labels are unavailable.
A practical KNN evaluation workflow
Reserve validation data before choosing scaling, features, k, weights, or a metric. Put preprocessing and KNN in one repeatable pipeline so every fold learns scaling only from its training portion. Evaluate more than overall accuracy when classes are imbalanced: per-class recall, precision, F1, a confusion matrix, and calibrated decision requirements can reveal failures hidden by an average score.
Inspect errors in feature space and ask whether nearby samples should genuinely share a label. Duplicate records can make validation look unrealistically good, while time-dependent or user-dependent rows may require grouped or chronological splits. At prediction time, monitor distance to the neighborhood as well as the winning label. A query far from every training observation is an extrapolation even if KNN returns a confident-looking majority. Production systems should define an abstention or fallback rule for such cases, measure latency as the reference set grows, and document which training records are retained. The playground isolates geometry; a dependable application adds data governance, representative evaluation, and monitoring around it.