BoWModel¶
bow_mod: A wrapper for Bag-of-Words models
Author: Evgenii Nikitin <e.nikitin@nyu.edu>
Part of https://github.com/crazyfrogspb/RedditScore project
Copyright (c) 2018 Evgenii Nikitin. All rights reserved. This work is licensed under the terms of the MIT license.
-
class
models.bow_mod.BoWModel(estimator, ngrams=1, tfidf=True, random_state=24)[source]¶ A wrapper for Bag-of-Words models with or without tf-idf re-weighting
Parameters: - estimator (scikit-learn model) – Estimator object (classifier or regressor)
- ngrams (int, optional) – The upper boundary of the range of n-values for different n-grams to be extracted
- tfidf (bool, optional) – If true, use tf-idf re-weighting
- random_state (integer, optional) – Random seed
- **kwargs – Parameters of the multinomial model. For details check scikit-learn documentation.
-
params¶ dict – Dictionary with model parameters
-
cv_score(X, y, cv=0.2, scoring='accuracy', k=3)¶ Calculate validation score
Parameters: - X (iterable, shape (n_samples, )) – Sequence of tokenized documents
- y (iterable, shape (n_samples, )) – Sequence of labels
- cv (float, int, cross-validation generator or an iterable, optional) –
Determines the cross-validation splitting strategy. Possible inputs for cv are:
- float, to use holdout set of this size
- None, to use the default 3-fold cross validation,
- integer, to specify the number of folds in a StratifiedKFold,
- An object to be used as a cross-validation generator.
- An iterable yielding train, test splits.
- scoring (string, callable or None, optional, optional) – A string (see sklearn model evaluation documentation) or a scorer callable object or ‘top_k_accuracy’
- k (int, optional) – k parameter for ‘top_k_accuracy’ scoring
Returns: Average value of the validation metrics
Return type: float
-
fit(X, y)¶ Fit model
Parameters: - X (iterable, shape (n_samples, )) – Sequence of tokenized documents
- y (iterable, shape (n_samples, )) – Sequence of labels
Returns: Fitted model object
Return type:
-
fit_transform(X, y=None, **fit_params)¶ Fit to data, then transform it.
Fits transformer to X and y with optional parameters fit_params and returns a transformed version of X.
Parameters: - X (numpy array of shape [n_samples, n_features]) – Training set.
- y (numpy array of shape [n_samples]) – Target values.
Returns: X_new – Transformed array.
Return type: numpy array of shape [n_samples, n_features_new]
-
get_params(deep=None)¶ Get parameters of the model
Returns: Dictionary with model parameters Return type: dict
-
plot_analytics(classes=None, fig_sizes=((20, 15), (20, 20)), linkage_pars=None, dendrogram_pars=None, clustering_pars=None, tsne_pars=None, legend_pars=None, label_font_size=17)¶ Plot hieracical clustering dendrogram and T-SNE visualization based on the learned class embeddings
Parameters: - classes (iter, optional) – Iterable, contains list of class labels to include to the plots. If None, use all classes
- fig_sizes (tuple of tuples, optional) – Figure sizes for plots
- linkage_pars (dict, optional) – Dictionary of parameters for hieracical clustering. (scipy.cluster.hierarchy.linkage)
- dendrogram_pars (dict, optional) – Dictionary of parameters for plotting dendrogram. (scipy.cluster.hierarchy.dendrogram)
- clustering_pars (dict, optional) – Dictionary of parameters for producing flat clusters. (scipy.cluster.hierarchy.fcluster)
- tsne_pars (dict, optional) – Dictionary of parameters for T-SNE. (sklearn.manifold.TSNE)
- legend_pars (dict, optional) – Dictionary of parameters for legend plotting (matplotlib.pyplot.legend)
- label_font_size (int, optional) – Font size for the labels on T-SNE plot
-
predict(X)¶ Predict the most likely label
Parameters: - X (iterable, shape (n_samples, )) – Sequence of tokenized documents
- y (iterable, shape (n_samples, )) – Sequence of labels
Returns: Predicted class labels
Return type: array, shape (n_samples, )
-
predict_proba(X)¶ Predict the most likely label
Parameters: - X (iterable, shape (n_samples, )) – Sequence of tokenized documents
- y (iterable, shape (n_samples, )) – Sequence of labels
Returns: Predicted class probabilities
Return type: array, shape (n_samples, num_classes)
-
save_model(filepath)[source]¶ Save model to disk.
Parameters: filepath (str) – Path to the file where the model will be sabed.
-
set_params(**params)[source]¶ Set the parameters of the model.
Parameters: **params ({'tfidf', 'ngrams', 'random_state'} or) – parameters of the corresponding models
-
tune_params(X, y, param_grid=None, verbose=False, cv=0.2, scoring='accuracy', k=3, refit=False)¶ Find the best values of hyperparameters using chosen validation scheme
Parameters: - X (iterable, shape (n_samples, )) – Sequence of tokenized documents
- y (iterable, shape (n_samples, )) – Sequence of labels
- param_grid (dict, optional) – Dictionary with parameters names as keys and lists of parameter settings as values. If None, loads deafult values from JSON file
- verbose (bool, optional) – If True, print scores after fitting each model
- cv (float, int, cross-validation generator or an iterable, optional) –
Determines the cross-validation splitting strategy. Possible inputs for cv are:
- float, to use holdout set of this size
- None, to use the default 3-fold cross validation,
- integer, to specify the number of folds in a StratifiedKFold,
- An object to be used as a cross-validation generator.
- An iterable yielding train, test splits.
- scoring (string, callable or None, optional) – A string (see sklearn model evaluation documentation) or a scorer callable object or ‘top_k_accuracy’
- k (int, optional) – k parameter for ‘top_k_accuracy’ scoring
- refit (boolean, optional) – If True, refit model with the best found parameters
Returns: - best_pars (dict) – Dictionary with the best combination of parameters
- best_value (float) – Best value of the chosen metric