StandardScaler#
- class ray.data.preprocessors.StandardScaler(columns: List[str], output_columns: List[str] | None = None)[source]#
Bases:
SerializablePreprocessorBaseTranslate and scale each column by its mean and standard deviation, respectively.
The general formula is given by
\[x' = \frac{x - \bar{x}}{s}\]where \(x\) is the column, \(x'\) is the transformed column, \(\bar{x}\) is the column average, and \(s\) is the column’s sample standard deviation. If \(s = 0\) (i.e., the column is constant-valued), then the transformed column will contain zeros.
Warning
StandardScalerworks best when your data is normal. If your data isn’t approximately normal, then the transformed features won’t be meaningful.Examples
>>> import pandas as pd >>> import ray >>> from ray.data.preprocessors import StandardScaler >>> >>> df = pd.DataFrame({"X1": [-2, 0, 2], "X2": [-3, -3, 3], "X3": [1, 1, 1]}) >>> ds = ray.data.from_pandas(df) >>> ds.to_pandas() X1 X2 X3 0 -2 -3 1 1 0 -3 1 2 2 3 1
Columns are scaled separately.
>>> preprocessor = StandardScaler(columns=["X1", "X2"]) >>> preprocessor.fit_transform(ds).to_pandas() X1 X2 X3 0 -1.224745 -0.707107 1 1 0.000000 -0.707107 1 2 1.224745 1.414214 1
Constant-valued columns get filled with zeros.
>>> preprocessor = StandardScaler(columns=["X3"]) >>> preprocessor.fit_transform(ds).to_pandas() X1 X2 X3 0 -2 -3 0.0 1 0 -3 0.0 2 2 3 0.0
>>> preprocessor = StandardScaler( ... columns=["X1", "X2"], ... output_columns=["X1_scaled", "X2_scaled"] ... ) >>> preprocessor.fit_transform(ds).to_pandas() X1 X2 X3 X1_scaled X2_scaled 0 -2 -3 1 -1.224745 -0.707107 1 0 -3 1 0.000000 -0.707107 2 2 3 1 1.224745 1.414214
- Parameters:
PublicAPI (alpha): This API is in alpha and may change before becoming stable.
Methods
Deserialize a preprocessor from serialized data.
Fit this Preprocessor to the Dataset.
Return the fit status of this preprocessor.
Fit this Preprocessor to the Dataset and then transform the Dataset.
Return the names of the columns this preprocessor reads.
Return the names of the columns this preprocessor writes.
Get the preprocessor class identifier for this preprocessor class.
Get the version number for this preprocessor class.
Return whether this preprocessor holds computed statistics.
DeveloperAPI: This API may change across minor Ray releases.
Serialize this preprocessor to a string or bytes.
Set the preprocessor class identifier for this preprocessor class.
Set the version number for this preprocessor class.
Transform the given dataset.
Transform a single batch of data.
Attributes
The columns to separately scale.
The names of the transformed columns.
The aggregations that
fit()runs to compute statistics.