Python sklearn Label Encoding and One Hot Encoding

When you pass categorical columns to a scikit-learn estimator, you will often encounter a ValueError about being unable to convert strings to float. That is because most sklearn models expect numeric input. The two most common ways to convert categorical text into numbers are label encoding and one-hot encoding, both available in sklearn.preprocessing. This article covers python sklearn label encoding and one hot encoding, how they differ, and how to choose between them.

What Label Encoding Does in sklearn

LabelEncoder maps each unique category to an integer. It is a simple ordinal encoding that assigns values like 0, 1, 2, ... based on alphabetical order by default.

from sklearn.preprocessing import LabelEncoder

colors = ['red', 'green', 'blue', 'green', 'red']
encoder = LabelEncoder()
encoded = encoder.fit_transform(colors)
print(encoded)  # [2, 1, 0, 1, 2]
print(encoder.classes_)  # ['blue' 'green' 'red']

The mapping is consistent: blue becomes 0, green becomes 1, red becomes 2. You can reverse the transformation with inverse_transform if you need to recover the original labels.

LabelEncoder is designed for target variables (y), not features. Using it on input features is common but problematic because it introduces an artificial ordering. For example, if you encode ['small', 'medium', 'large'] as [0, 1, 2], the model may treat medium as halfway between small and large, which is only valid if the categories are ordinal. For nominal categories like colors, this ordering is meaningless and can degrade model performance.

What One-Hot Encoding Does in sklearn

OneHotEncoder creates a binary column for each category. Each row gets a 1 in the column corresponding to its category and 0 elsewhere. This avoids imposing any ordinal relationship.

from sklearn.preprocessing import OneHotEncoder
import numpy as np

colors = np.array(['red', 'green', 'blue', 'green', 'red']).reshape(-1, 1)
encoder = OneHotEncoder(sparse_output=False)
encoded = encoder.fit_transform(colors)
print(encoded)
# [[0. 0. 1.]
#  [0. 1. 0.]
#  [1. 0. 0.]
#  [0. 1. 0.]
#  [0. 0. 1.]]
print(encoder.categories_)  # [array(['blue', 'green', 'red'], dtype=object)]

By default, OneHotEncoder returns a sparse matrix to save memory. Set sparse_output=False if you need a dense NumPy array. The encoder also supports drop='first' to avoid multicollinearity in linear models, and handle_unknown='ignore' to tolerate categories that were not present during training.

Unlike LabelEncoder, OneHotEncoder is designed for feature matrices. It can handle multiple columns at once and returns a single matrix with all encoded columns concatenated.

Key Differences Between Label Encoding and One-Hot Encoding

The two encoders differ in output shape, interpretability, and the assumptions they make about the data.

Label encoding is compact and keeps the feature count unchanged, but it leaks ordinal information. One-hot encoding is safer for nominal data but can explode the feature space when a column has many unique values.

When to Use Label Encoding vs One-Hot Encoding

Use label encoding when the categorical variable is ordinal and the order carries meaning. For example, education level (high school, bachelor, master, phd) or customer satisfaction (low, medium, high). In these cases, the integer mapping preserves the natural ordering and can help tree-based models split more efficiently.

Use one-hot encoding when the categories are nominal and have no intrinsic order. Colors, countries, product types, and most string identifiers fall into this group. One-hot encoding ensures the model does not infer a false ranking.

There is also a practical middle ground: if a nominal column has a very high cardinality (e.g., thousands of categories), one-hot encoding becomes impractical. In that situation, you might use target encoding or frequency encoding instead, but those are outside the scope of this article. For most categorical features with fewer than a few dozen categories, one-hot encoding is the default choice.

Handling New Categories and Unseen Values

A common production issue is encountering a category during inference that was not present during training. LabelEncoder has no built-in mechanism for this; it will raise an error if you call transform on an unseen label. You would need to manually map unknown values to a fallback integer, which is brittle.

OneHotEncoder handles this more gracefully. Set handle_unknown='ignore' to produce an all-zero row for unseen categories. This prevents crashes but also means the model receives no signal for that category. If you want to explicitly model unknown values, you can add a dedicated category during training, such as "<unknown>", and replace unseen values with it before transforming.

encoder = OneHotEncoder(handle_unknown='ignore', sparse_output=False)
encoder.fit(np.array(['red', 'green']).reshape(-1, 1))
print(encoder.transform(np.array(['blue']).reshape(-1, 1)))
# [[0. 0.]]  # all zeros because 'blue' was not seen

This behavior is important for maintaining stable model pipelines when new data arrives over time.

Performance and Memory Considerations

One-hot encoding increases the number of columns, which directly affects memory usage and training time. A categorical column with 50 unique values becomes 50 binary columns. If you have several high-cardinality columns, the feature matrix can grow large. The sparse matrix representation helps, but tree-based models like random forests often require dense arrays, so you may need to convert to dense and pay the memory cost.

Label encoding avoids this growth but introduces a different problem: the model may interpret the integers as continuous values. Linear models and distance-based algorithms like k-nearest neighbors are especially sensitive to this. For tree-based models, label encoding on nominal data can still work because trees can split on arbitrary thresholds, but the ordinal assumption can still bias the split selection.

If memory is a constraint and the categorical variable is nominal, consider using OneHotEncoder with drop='first' to reduce one column per category, or group rare categories into an "other" bucket before encoding.

Integrating Encoders into a Scikit-Learn Pipeline

In practice, you rarely apply a single encoder to a whole dataset. You often have a mix of numeric and categorical columns, and you want to apply different transformations to each. ColumnTransformer lets you do this cleanly.

from sklearn.compose import ColumnTransformer
from sklearn.preprocessing import OneHotEncoder, StandardScaler
from sklearn.pipeline import Pipeline
from sklearn.linear_model import LogisticRegression
import pandas as pd

df = pd.DataFrame({
    'age': [25, 30, 35],
    'city': ['NYC', 'LA', 'SF'],
    'salary': [70000, 80000, 90000]
})

preprocessor = ColumnTransformer(
    transformers=[
        ('num', StandardScaler(), ['age', 'salary']),
        ('cat', OneHotEncoder(handle_unknown='ignore'), ['city'])
    ]
)

pipeline = Pipeline(steps=[
    ('preprocessor', preprocessor),
    ('classifier', LogisticRegression())
])

pipeline.fit(df, [0, 1, 0])

This pipeline ensures that the encoder is fitted only on training data and applied consistently to test data. It also makes cross-validation safe because the encoding is part of the model fitting process, preventing data leakage.

For label encoding, you can use OrdinalEncoder from sklearn.preprocessing instead of LabelEncoder when you need to encode multiple feature columns. OrdinalEncoder works like LabelEncoder but accepts a 2D array and is designed for features.

from sklearn.preprocessing import OrdinalEncoder

X = [['low'], ['medium'], ['high']]
encoder = OrdinalEncoder(categories=[['low', 'medium', 'high']])
print(encoder.fit_transform(X))
# [[0.]
#  [1.]
#  [2.]]

When you need to preserve the exact category order, pass the categories parameter explicitly. This also makes the encoder robust to unseen categories if you set handle_unknown='use_encoded_value' with a fallback value.

Choosing between label encoding and one-hot encoding is not a one-size-fits-all decision. It depends on the nature of your categorical data, the model you plan to use, and the operational constraints of your pipeline. For nominal features, one-hot encoding is the safer default; for ordinal features, label encoding (or OrdinalEncoder) preserves meaningful order. By understanding the tradeoffs and using ColumnTransformer, you can build preprocessing steps that are both correct and maintainable.