A hard margin refers to a specific approach used in support vector machines (SVM) for classification tasks in machine learning. In the context of SVMs, a hard margin is characterized by the requirement that all training data points must be correctly classified with a clear margin, meaning there should be no misclassifications. This approach is applicable when the data is linearly separable.
In practical terms, a hard margin SVM finds a hyperplane that separates data points of different classes while maximizing the distance (or margin) between the nearest points of each class, known as support vectors. The larger this margin, the better the generalization performance of the model is expected to be, as it implies a more robust separation between classes.
However, the hard margin approach has its limitations. It can be overly sensitive to outliers and noise in the dataset, as the presence of even a single misclassified point can prevent the model from finding a suitable hyperplane. Due to this sensitivity, hard margin SVMs are typically used in scenarios where the data is clean and well-separated. In cases where data is noisy or not perfectly separable, a soft margin SVM is often preferred, allowing for some misclassifications while still maintaining a balance between margin width and classification accuracy.