What is Splunk RobustScaler, and how is it different from StandardScaler
Splunk RobustScaler is an essential preprocessing tool that comes in handy when dealing with data that contains outliers. Splunk MLTK uses the RobustScaler preprocessing algorithm from the scikit-learn Python library. The role of this scaler is to adjust the features in your dataset in a way less affected by extreme values compared to the traditional StandardScaler.
Affiliate: Experience limitless no-code automation, streamline your workflows, and effortlessly transfer data between apps with Make.com.
While both Splunk RobustScaler and StandardScaler are standardizing numeric field algorithms, they operate on different statistical measures. The StandardScaler focuses on the mean and standard deviation, setting them to 0 and 1, respectively. On the other hand, Splunk RobustScaler works with the median and the interquartile range (IQR), putting their values to 0 and 1, respectively. The output of the RobustScaler will transform between negative and positive points, while “0” will be the center.
This unique approach of the Splunk RobustScaler makes it more resilient to outliers. Outliers can massively inflate the mean of a dataset, which can significantly affect the results of standard scaling. However, using the median, which remains unchanged even when outliers are present, makes Splunk RobustScaler a more reliable tool in such scenarios.
Further, outliers can cause the mean and standard deviation to spike, leading to a distorted representation of the original data distribution when using StandardScaler. Splunk RobustScaler, with its use of the median and IQR, maintains the relative distance between outliers and other values, offering a more accurate representation of the data distribution.
Therefore, when your input data contains outliers, Splunk RobustScaler is recommended. Its resilience to the adverse effects of outliers, thanks to its use of the median and IQR, ensures it provides a more accurate scaling of your data.
If you need help installing Machine Learning Toolkit in Splunk, consider these articles: How to Install Splunk Addons and How to Prepare Splunk for Machine Learning.
RobustScaler SPL Usage
The usage of Splunk RobustScaler is similar to StandardScaler in MLTK:
fit RobustScaler field1 field2 field3 with_centering=true with_scaling=true into model_nameIn this command:
fit RobustScaler field1 field2 field3: This is the basic command to fit the RobustScaler to your chosen fields. Replace “field1 field2 field3” with the names of the fields you want to scale. Each field will scale separately.
into model_name: This argument is optional. Use this to save the resulting model with a specific name. Replace “model_name” with your chosen name for the model. If you don’t want to keep it to any model but only use it for scaling the input data points, this part can be omitted, precisely as in Machine Learning Toolkit StandardScaler.
with_centering: This argument is also optional. Set this to “true” if you want to center the data before scaling (i.e., subtract the median). If not, set this to “false.”
with_scaling: Another optional argument. Set this to “true” if you want to scale the data (i.e., divide by the interquartile range). If not, set this to “false.”
The command will create new fields in your data named “RS_field1”, “RS_field2”, and “RS_field3,” which contain the scaled values of the fields “field1”, “field2”, and “field3,” respectively.
If you want to fit Splunk RobustScaler for every specific interval, consider using standard scaling with built-in Splunk functions in the “Splunk Standard Scaling: ML Preprocessing, Normalization” article.