{"id":377,"date":"2022-10-24T14:10:23","date_gmt":"2022-10-24T11:10:23","guid":{"rendered":"https:\/\/quecst.qcri.org\/blog\/?p=377"},"modified":"2022-10-24T14:10:23","modified_gmt":"2022-10-24T11:10:23","slug":"log-odds-ratio-going-beyond-simple-term-frequencies-to-characterize-textual-categories","status":"publish","type":"post","link":"https:\/\/acua.qcri.org\/blog\/log-odds-ratio-going-beyond-simple-term-frequencies-to-characterize-textual-categories\/","title":{"rendered":"Log Odds Ratio: Going Beyond Simple Term Frequencies to Characterize Textual Categories"},"content":{"rendered":"

Gaining insights from text-based data can be a daunting task, even when the data is labeled with ground truth categories and ready for usage in machine learning tasks.
Researchers often rely on simple methods like the frequency of words in each category to understand the collection\u2019s characteristics. However, this approach is not always insightful, as term frequencies alone are not enough to distinguish between the categories because they fail to consider the fundamental differences between categories.<\/p>\n

\"\"
The issue of getting meaningful insights from textual and categorical data (source of image<\/a>)<\/figcaption><\/figure>\n

Therefore, in this tutorial, we will teach you how to use Log Odds Ratio, which is an alternative method to term frequency, TFIDF, and other term ranking techniques to obtain unique insights about the terms that represent categories in textual data [1].<\/p>\n

What is Log Odds Ratio?<\/strong><\/span><\/p>\n

Let\u2019s start with what is meant by odds:<\/p>\n

\"\"<\/p>\n

Keep in mind that the odds can be a very small number.
So now, the odds ratio is the ratio of the odds of a term being used in one category vs. another.
It can be computed using the following equation:
\"\"<\/p>\n

The Log-Odds-Ratio is often used, given these quantities are often very small. For our purposes, the fact that it’s normally distributed is important.
\"\"<\/p>\n

If term does not appear in category b (i.e., \ud835\udc66\ud835\udc4f\ud835\udc56=0), the odd of termi in categoryb will be 0 resulting the odds-ratio to be undefined. We circumvent this issue by adding a pseudo-count to each term, assuming it occurs at least \ud835\udefc times.
This will result in an equation that computes the smoothed Log-Odds-Ratio as follows:
\"\"
This has a slightly better profile than TFIDF, but still favors very low-frequency terms that are unique to a category.
The smoothed Log-Odds-Ratio ranks terms slightly better than TFIDF, yet it still prefers very low frequent terms which are unique to a category.
Note that when terms are exclusive to one category or another, their cluster can be extremely high or low scoring compared to others because these terms occur infrequently. Intuitively, we’d like to score things highly if they have a lot of evidence behind them AND they tend to be used much more in one category vs. another. However, in cases where the dataset has terms that appear in both categories (i.e., a lot of intersecting terms, even after cleaning), the Log Odds Ratio can be a suitable method of ranking terms to gain insights on the type of content in each category.<\/p>\n

For this tutorial, we will use a random sample of a dataset that was used in [2] to build a machine learning model for detecting toxicity triggers (i.e., causes of toxicity in online conversation threads). Furthermore, we will use scattertext [3], which is a powerful text visualization library that generates interactive scatter plots of corpora and renders them via html. For preprocessing the dataset, we will use spacy [4], which is a powerful open source NLP library that helps with different types of NLP-based text processing tasks.<\/p>\n

The dataset is available for download from this link<\/a>.<\/p>\n

After downloading the dataset, make sure that you install the following packages:<\/p>\n

#Step 1: download missing libraries into Google colab with the pip command<\/span>\r\n!pip install -<\/span>-<\/span>quiet scattertext\r\n!pip install -<\/span>-<\/span>quiet spacy\r\n!python3 -<\/span>m spacy download en_core_web_sm\r\n<\/pre>\n
<\/p>\n
Next, import the necessary libraries and download the required components for performing NLP tasks, as follows:<\/p>\n
#Step 2: import the required libraries or load the needed modules<\/span>\r\nimport<\/span> re\r\nimport<\/span> pandas as<\/span> pd\r\nimport<\/span> numpy as<\/span> np\r\nimport<\/span> scattertext as<\/span> st\r\nimport<\/span> spacy\r\nfrom<\/span> IPython.<\/span>display import<\/span> IFrame\r\nfrom<\/span> IPython.<\/span>core.<\/span>display import<\/span> display,<\/span> HTML\r\nimport<\/span> html\r\ndisplay(<\/span>HTML(<\/span>\"<style>.container { width:98% !important; }<\/style>\"<\/span>)<\/span>)<\/span> #For visualization<\/span>\r\nnlp =<\/span> spacy.<\/span>load(<\/span>\"en_core_web_sm\"<\/span>)<\/span>\r\nfrom<\/span> bs4 import<\/span> BeautifulSoup\r\nimport<\/span> nltk\r\nfrom<\/span> nltk.<\/span>corpus import<\/span> stopwords\r\nnltk.<\/span>download(<\/span>'stopwords'<\/span>)<\/span>\r\nimport<\/span> warnings\r\nwarnings.<\/span>filterwarnings(<\/span>'ignore'<\/span>)<\/span>\r\n<\/pre>\n
<\/p>\n
Then, make a function for cleaning the raw text from the dataset and another one to map the categories to a numerical value as follows:<\/p>\n
#Step 3: function to convert the comments to words<\/span>\r\ndef<\/span> comment_to_words(<\/span> raw_comment )<\/span>:<\/span>\r\n    # 1. Remove HTML<\/span>\r\n    comment_text =<\/span> BeautifulSoup(<\/span>raw_comment,<\/span> 'lxml'<\/span>)<\/span>.<\/span>get_text(<\/span>)<\/span> \r\n    # 2. Remove non-letters with regex<\/span>\r\n    letters_only =<\/span> re.<\/span>sub(<\/span>\"[^a-zA-Z]\"<\/span>,<\/span> \" \"<\/span>,<\/span> comment_text)<\/span>   \r\n    # 3. Convert to lower case, split into individual words<\/span>\r\n    words =<\/span> letters_only.<\/span>lower(<\/span>)<\/span>.<\/span>split(<\/span>)<\/span>                              \r\n    # 4. Create set of stopwords<\/span>\r\n    stops =<\/span> set<\/span>(<\/span>stopwords.<\/span>words(<\/span>\"english\"<\/span>)<\/span>)<\/span>                    \r\n    # 5. Remove stop words<\/span>\r\n    meaningful_words =<\/span> [<\/span>w for<\/span> w in<\/span> words if<\/span> not<\/span> w in<\/span> stops]<\/span>      \r\n    # 6. Join the words back into one string separated by space, <\/span>\r\n    # and return the result.<\/span>\r\n    return<\/span>(<\/span> \" \"<\/span>.<\/span>join(<\/span> meaningful_words )<\/span>)<\/span>\r\n\r\n#A function to change the category from numerical to text<\/span>\r\ndef<\/span> get_category(<\/span>row)<\/span>:<\/span>\r\n    for<\/span> c in<\/span> trigger_categories:<\/span>\r\n        if<\/span> row[<\/span>c]<\/span> ==<\/span> 1<\/span>:<\/span>\r\n            return<\/span> 'Trigger'<\/span>\r\n    return<\/span> 'Not-trigger'<\/span>\r\n    \r\ndef<\/span> get_category_list(<\/span>row)<\/span>:<\/span>\r\n    return<\/span> [<\/span>c for<\/span> c in<\/span> trigger_categories if<\/span> row[<\/span>c]<\/span> ==<\/span> 1<\/span>]<\/span>   \r\n<\/pre>\n
<\/p>\n
After that, read the dataset and perform the preprocessing task:<\/p>\n
#Step 4: read data and preprocess it<\/span>\r\ndf =<\/span> pd.<\/span>read_csv(<\/span>'toxicity_triggers_sample.csv'<\/span>)<\/span>\r\n\r\n#print dataset header before preprocessing <\/span>\r\nprint<\/span>(<\/span>\"Before preprocessing:\"<\/span>)<\/span>\r\nprint<\/span>(<\/span>df.<\/span>head(<\/span>)<\/span>)<\/span>\r\n\r\n#Create additional columns with the numerical representations of labels<\/span>\r\ndf[<\/span>'Trigger'<\/span>]<\/span> =<\/span> 0<\/span>\r\ndf[<\/span>'Trigger'<\/span>]<\/span>.<\/span>loc[<\/span>df[<\/span>'category'<\/span>]<\/span> ==<\/span> 'trigger'<\/span>]<\/span> =<\/span> 1<\/span>\r\ndf[<\/span>'NotTrigger'<\/span>]<\/span> =<\/span> 0<\/span>\r\ndf[<\/span>'NotTrigger'<\/span>]<\/span>.<\/span>loc[<\/span>df[<\/span>'category'<\/span>]<\/span> ==<\/span> 'non_trigger'<\/span>]<\/span> =<\/span> 0<\/span>\r\n\r\n#Delete the category column<\/span>\r\ndel<\/span> df[<\/span>'category'<\/span>]<\/span>\r\n\r\n#Define the list of categories <\/span>\r\ntrigger_categories =<\/span> [<\/span>'Trigger'<\/span>,<\/span>'NotTrigger'<\/span>]<\/span>\r\n\r\ndf[<\/span>'category'<\/span>]<\/span> =<\/span> df.<\/span>apply(<\/span>get_category,<\/span> axis=<\/span>1<\/span>)<\/span>\r\ndf[<\/span>'category_list'<\/span>]<\/span> =<\/span> df.<\/span>apply(<\/span>get_category_list,<\/span> axis=<\/span>1<\/span>)<\/span>\r\n\r\n#Print the count of triggers and non-triggers in the dataset<\/span>\r\nprint<\/span>(<\/span>df.<\/span>category.<\/span>value_counts(<\/span>)<\/span>)<\/span>\r\n\r\n#Fill nan or missing values with _na_<\/span>\r\ndf[<\/span>'Text'<\/span>]<\/span>=<\/span> df[<\/span>'Text'<\/span>]<\/span>.<\/span>fillna(<\/span>\"_na_\"<\/span>)<\/span>.<\/span>values\r\ndf[<\/span>'Processed_Text'<\/span>]<\/span> =<\/span>df[<\/span>'Text'<\/span>]<\/span>.<\/span>apply(<\/span>comment_to_words)<\/span>\r\ndf[<\/span>'parse'<\/span>]<\/span> =<\/span> df.<\/span>Processed_Text.<\/span>apply(<\/span>nlp)<\/span>\r\n\r\n#Print dataset header after preprocessing<\/span>\r\nprint<\/span>(<\/span>\"After preprocessing:\"<\/span>)<\/span>\r\nprint<\/span>(<\/span>df.<\/span>head(<\/span>)<\/span>)<\/span>\r\n<\/pre>\n

\nNow, use Log Odds Ratio (with Dirichlet Prior) to know which ranked terms appear the most in the dataset:<\/p>\n
#Step 5: Use scattertext to create a corpus from parsed documents, use log odds ratio to rank and print top 10, 30, and 50 terms per category<\/span>\r\n#Note that the terms displayed here are not using any smoothing in the LOR, instead it uses Informative Dirichlet Prior<\/span>\r\ncorpus =<\/span> st.<\/span>CorpusFromParsedDocuments(<\/span>\r\n    df,<\/span> \r\n    parsed_col=<\/span>'parse'<\/span>,<\/span> \r\n    category_col=<\/span>'category'<\/span>,<\/span> \r\n    feats_from_spacy_doc=<\/span>st.<\/span>UnigramsFromSpacyDoc(<\/span>)<\/span>)<\/span>.<\/span>build(<\/span>)<\/span>\r\n\r\n#Print the total number of terms in the corpus<\/span>\r\nprint<\/span>(<\/span>len<\/span>(<\/span>corpus.<\/span>get_terms(<\/span>)<\/span>)<\/span>)<\/span>\r\nfor<\/span> i in<\/span> range<\/span>(<\/span>1<\/span>,<\/span> 20<\/span>)<\/span>:<\/span>\r\n    print<\/span>(<\/span>'Threshold:'<\/span>,<\/span> i,<\/span> '# terms:'<\/span>,<\/span> \r\n          len<\/span>(<\/span>corpus.<\/span>compact(<\/span>st.<\/span>ClassPercentageCompactor(<\/span>st.<\/span>OncePerDocFrequencyRanker,<\/span> i)<\/span>)<\/span>.<\/span>get_terms(<\/span>)<\/span>)<\/span>)<\/span>\r\n\r\n\r\ncompact_corpus =<\/span> corpus.<\/span>compact(<\/span>st.<\/span>ClassPercentageCompactor(<\/span>st.<\/span>OncePerDocFrequencyRanker,<\/span> 14<\/span>)<\/span>)<\/span>\r\ndfs =<\/span> [<\/span>]<\/span>\r\n\r\nsingle_df_columns =<\/span> [<\/span>'parse'<\/span>,<\/span> 'category_list'<\/span>,<\/span> 'single_category'<\/span>]<\/span>\r\nfor<\/span> cat in<\/span> trigger_categories:<\/span>\r\n    #print(cat)<\/span>\r\n    new_df =<\/span> df[<\/span>(<\/span>df[<\/span>cat]<\/span> ==<\/span> 1<\/span>)<\/span>]<\/span>\r\n    new_df[<\/span>'single_category'<\/span>]<\/span> =<\/span> cat\r\n    dfs.<\/span>append(<\/span>new_df[<\/span>single_df_columns]<\/span>)<\/span>\r\nnew_df =<\/span> df[<\/span>df[<\/span>trigger_categories]<\/span>.<\/span>sum<\/span>(<\/span>axis=<\/span>1<\/span>)<\/span> ==<\/span> 0<\/span>]<\/span>\r\nnew_df[<\/span>'single_category'<\/span>]<\/span> =<\/span> 'Not-trigger'<\/span>\r\ndfs.<\/span>append(<\/span>new_df[<\/span>single_df_columns]<\/span>)<\/span>\r\nsingle_df =<\/span> pd.<\/span>concat(<\/span>dfs)<\/span>\r\ndel<\/span> dfs\r\nsingle_category_corpus =<\/span> st.<\/span>CorpusFromParsedDocuments(<\/span>\r\n    single_df,<\/span> \r\n    parsed_col=<\/span>'parse'<\/span>,<\/span> \r\n    category_col=<\/span>'single_category'<\/span>,<\/span> \r\n    feats_from_spacy_doc=<\/span>st.<\/span>UnigramsFromSpacyDoc(<\/span>)<\/span>)<\/span>.<\/span>build(<\/span>)<\/span>\r\nterm_freq_df =<\/span> st.<\/span>OncePerDocFrequencyRanker(<\/span>single_category_corpus)<\/span>.<\/span>get_ranks(<\/span>)<\/span>\r\n\r\nfor<\/span> c in<\/span> single_category_corpus.<\/span>get_categories(<\/span>)<\/span>:<\/span>\r\n    \r\n    scores =<\/span>  (<\/span>st.<\/span>ScaledFScorePresets(<\/span>beta =<\/span> 1<\/span>,<\/span> one_to_neg_one=<\/span>True<\/span>)<\/span>\r\n               .<\/span>get_scores(<\/span>\r\n                   term_freq_df[<\/span>c +<\/span> ' freq'<\/span>]<\/span>,<\/span> \r\n                   term_freq_df[<\/span>[<\/span>oc for<\/span> oc in<\/span> term_freq_df.<\/span>columns \r\n                                 if<\/span> oc !=<\/span> c +<\/span> ' freq'<\/span> and<\/span> oc.<\/span>endswith(<\/span>' freq'<\/span>)<\/span>]<\/span>]<\/span>.<\/span>sum<\/span>(<\/span>axis=<\/span>1<\/span>)<\/span>\r\n               )<\/span>)<\/span>\r\n    term_freq_df[<\/span>'score'<\/span>]<\/span> =<\/span> scores\r\n    #Print the category and the ranked terms that + belong to the category or - don't<\/span>\r\n    print<\/span>(<\/span>c)<\/span>\r\n    print<\/span>(<\/span>\"10\"<\/span>)<\/span>\r\n    print<\/span>(<\/span>'+'<\/span>,<\/span>list<\/span>(<\/span>term_freq_df.<\/span>sort_values(<\/span>by=<\/span>'score'<\/span>,<\/span> ascending=<\/span>False<\/span>)<\/span>.<\/span>iloc[<\/span>:<\/span>10<\/span>]<\/span>.<\/span>index)<\/span>)<\/span>\r\n    print<\/span>(<\/span>'-'<\/span>,<\/span>list<\/span>(<\/span>term_freq_df.<\/span>sort_values(<\/span>by=<\/span>'score'<\/span>,<\/span> ascending=<\/span>True<\/span>)<\/span>.<\/span>iloc[<\/span>:<\/span>10<\/span>]<\/span>.<\/span>index)<\/span>)<\/span>   \r\n    print<\/span>(<\/span>\"30\"<\/span>)<\/span>\r\n    print<\/span>(<\/span>'+'<\/span>,<\/span>list<\/span>(<\/span>term_freq_df.<\/span>sort_values(<\/span>by=<\/span>'score'<\/span>,<\/span> ascending=<\/span>False<\/span>)<\/span>.<\/span>iloc[<\/span>:<\/span>30<\/span>]<\/span>.<\/span>index)<\/span>)<\/span>\r\n    print<\/span>(<\/span>'-'<\/span>,<\/span>list<\/span>(<\/span>term_freq_df.<\/span>sort_values(<\/span>by=<\/span>'score'<\/span>,<\/span> ascending=<\/span>True<\/span>)<\/span>.<\/span>iloc[<\/span>:<\/span>30<\/span>]<\/span>.<\/span>index)<\/span>)<\/span> \r\n    print<\/span>(<\/span>\"50\"<\/span>)<\/span>\r\n    print<\/span>(<\/span>'+'<\/span>,<\/span>list<\/span>(<\/span>term_freq_df.<\/span>sort_values(<\/span>by=<\/span>'score'<\/span>,<\/span> ascending=<\/span>False<\/span>)<\/span>.<\/span>iloc[<\/span>:<\/span>50<\/span>]<\/span>.<\/span>index)<\/span>)<\/span>\r\n    print<\/span>(<\/span>'-'<\/span>,<\/span>list<\/span>(<\/span>term_freq_df.<\/span>sort_values(<\/span>by=<\/span>'score'<\/span>,<\/span> ascending=<\/span>True<\/span>)<\/span>.<\/span>iloc[<\/span>:<\/span>50<\/span>]<\/span>.<\/span>index)<\/span>)<\/span>\r\ntox_obs_corpus =<\/span> single_category_corpus\r\n\r\ntox_obs_corpus =<\/span> tox_obs_corpus.<\/span>compact(<\/span>st.<\/span>ClassPercentageCompactor(<\/span>term_count=<\/span>2<\/span>)<\/span>)<\/span>\r\npriors =<\/span> (<\/span>st.<\/span>PriorFactory(<\/span>single_category_corpus,<\/span> \r\n            category=<\/span>'Trigger'<\/span>,<\/span> \r\n            not_categories=<\/span>[<\/span>'Not-trigger'<\/span>]<\/span>)<\/span>\r\n      .<\/span>use_neutral_categories(<\/span>)<\/span>\r\n      .<\/span>align_to_target(<\/span>tox_obs_corpus)<\/span>\r\n      .<\/span>get_priors(<\/span>)<\/span>)<\/span>\r\nterm_ranker =<\/span> st.<\/span>OncePerDocFrequencyRanker\r\nterm_scorer =<\/span> st.<\/span>LogOddsRatioInformativeDirichletPrior(<\/span>priors,<\/span> scale_type=<\/span>'class-size'<\/span>,<\/span> sigma=<\/span>10<\/span>)<\/span>\r\nrank_df =<\/span> term_ranker(<\/span>tox_obs_corpus)<\/span>.<\/span>get_ranks(<\/span>)<\/span>\r\n#Perform an overall rank using LORIDP of the frequent trigger and non-trigger terms<\/span>\r\nrank_df[<\/span>'score'<\/span>]<\/span> =<\/span> term_scorer.<\/span>get_scores(<\/span>rank_df[<\/span>'Trigger freq'<\/span>]<\/span>,<\/span> rank_df[<\/span>'Not-trigger freq'<\/span>]<\/span>)<\/span>\r\nprint<\/span>(<\/span>'Trigger LORIDP 10'<\/span>,<\/span>list<\/span>(<\/span>rank_df.<\/span>sort_values(<\/span>by=<\/span>'score'<\/span>,<\/span> ascending=<\/span>False<\/span>)<\/span>.<\/span>iloc[<\/span>:<\/span>10<\/span>]<\/span>.<\/span>index)<\/span>)<\/span>\r\nprint<\/span>(<\/span>'NotTrigger LORIDP 10'<\/span>,<\/span>list<\/span>(<\/span>rank_df.<\/span>sort_values(<\/span>by=<\/span>'score'<\/span>,<\/span> ascending=<\/span>True<\/span>)<\/span>.<\/span>iloc[<\/span>:<\/span>10<\/span>]<\/span>.<\/span>index)<\/span>)<\/span>   \r\nprint<\/span>(<\/span>'Trigger LORIDP 30'<\/span>,<\/span>list<\/span>(<\/span>rank_df.<\/span>sort_values(<\/span>by=<\/span>'score'<\/span>,<\/span> ascending=<\/span>False<\/span>)<\/span>.<\/span>iloc[<\/span>:<\/span>30<\/span>]<\/span>.<\/span>index)<\/span>)<\/span>\r\nprint<\/span>(<\/span>'NotTrigger LORIDP 30'<\/span>,<\/span>list<\/span>(<\/span>rank_df.<\/span>sort_values(<\/span>by=<\/span>'score'<\/span>,<\/span> ascending=<\/span>True<\/span>)<\/span>.<\/span>iloc[<\/span>:<\/span>30<\/span>]<\/span>.<\/span>index)<\/span>)<\/span>   \r\nprint<\/span>(<\/span>'Trigger LORIDP 50'<\/span>,<\/span>list<\/span>(<\/span>rank_df.<\/span>sort_values(<\/span>by=<\/span>'score'<\/span>,<\/span> ascending=<\/span>False<\/span>)<\/span>.<\/span>iloc[<\/span>:<\/span>50<\/span>]<\/span>.<\/span>index)<\/span>)<\/span>\r\nprint<\/span>(<\/span>'NotTrigger LORIDP 50'<\/span>,<\/span>list<\/span>(<\/span>rank_df.<\/span>sort_values(<\/span>by=<\/span>'score'<\/span>,<\/span> ascending=<\/span>True<\/span>)<\/span>.<\/span>iloc[<\/span>:<\/span>50<\/span>]<\/span>.<\/span>index)<\/span>)<\/span>\r\n<\/pre>\n
<\/p>\n
Part of the output looks as follows:
\"\"
This outcome is similar to what we found in [2], which states that named entities like \u2018trump\u2019 are triggering terms, while terms that represent a positive action like \u2018workout\u2019 are not triggers.<\/p>\n
Well, what about using Log Odds Ratio with smoothing? Let\u2019s try it and visualize it as follows:<\/p>\n
#Step 6: use scattertext to plot the Log Odds Ratio with smoothing factor <\/span>\r\nclass<\/span> LogOddsRatioSmoothed:<\/span>\r\n    def<\/span> __init__<\/span>(<\/span>self,<\/span> alpha)<\/span>:<\/span>\r\n        self.<\/span>alpha =<\/span> alpha\r\n    def<\/span> get_scores(<\/span>self,<\/span> a,<\/span> b)<\/span>:<\/span> \r\n        return<\/span> (<\/span>np.<\/span>log(<\/span>(<\/span>a +<\/span> self.<\/span>alpha)<\/span>\/<\/span>(<\/span>np.<\/span>sum<\/span>(<\/span>a)<\/span> +<\/span> self.<\/span>alpha *<\/span> len<\/span>(<\/span>a)<\/span> -<\/span> a -<\/span> self.<\/span>alpha)<\/span>)<\/span> \r\n                -<\/span> np.<\/span>log(<\/span>(<\/span>b +<\/span> self.<\/span>alpha)<\/span>\/<\/span>(<\/span>np.<\/span>sum<\/span>(<\/span>b)<\/span> +<\/span> self.<\/span>alpha *<\/span> len<\/span>(<\/span>b)<\/span> -<\/span> b -<\/span> self.<\/span>alpha)<\/span>)<\/span>)<\/span>\r\n    def<\/span> get_name(<\/span>self)<\/span>:<\/span> \r\n        return<\/span> 'Smoothed Log-Odds-Ratio'<\/span>\r\n    \r\nhtml =<\/span> st.<\/span>produce_fightin_words_explorer(<\/span>\r\n    corpus,<\/span>\r\n    category=<\/span>'Trigger'<\/span>,<\/span>\r\n    not_category_name=<\/span>'Not-trigger'<\/span>,<\/span>\r\n    not_categories=<\/span>[<\/span>'Not-trigger'<\/span>]<\/span>,<\/span>\r\n    term_scorer=<\/span>LogOddsRatioSmoothed(<\/span>0.00000001<\/span>)<\/span>,<\/span>\r\n    grey_threshold=<\/span>0<\/span>,<\/span>\r\n)<\/span>\r\n\r\n#Save the html visualization file for later view<\/span>\r\nfile_name =<\/span> 'trigger_not_lorsmooth.html'<\/span>\r\nopen<\/span>(<\/span>file_name,<\/span> 'wb'<\/span>)<\/span>.<\/span>write(<\/span>html.<\/span>encode(<\/span>'utf-8'<\/span>)<\/span>)<\/span>\r\nHTML(<\/span>html)<\/span> #View the file<\/span>\r\n<\/pre>\n
<\/p>\n
The result of the visualization is as follows:
\"\"
Notice that this time we have different terms as triggers like \u2018Lebron\u2019 (another named entity that we found triggering in [2]). Furthermore, the top not-trigger terms from the above visualization show that terms related to technology like \u2018Computers\u2019 are not triggers. This finding also coincides with our work from [2], noting similarities between the discovered triggering and non-triggering terms.<\/p>\n
As final notes, we would like to emphasize that we do not claim that Log Odds Ratio is the best method to gain insights from text, as sometimes the least frequent terms may not be as informative or useful as we think. Therefore, we recommend trying other methods [1] on your dataset to see which term ranking technique produces the most useful insights for your analysis.<\/p>\n
The entire script is available on Google Colab<\/a>.<\/p>\n
Happy coding!<\/p>\n
References:<\/strong>
[1] \u201cJasonKessler\/SemioticSquaresTalk: #ddtx18 talk: Lexicon Mining for Semiotic Squares: Exploding Binary Classification.\u201d https:\/\/github.com\/JasonKessler\/SemioticSquaresTalk (accessed Oct. 19, 2022).
[2] H. Almerekhi, H. Kwak, J. Salminen, and B. J. Jansen, \u201cPROVOKE: Toxicity trigger detection in conversations from the top 100 subreddits,\u201d Data and Information Management, p. 100019, 2022.
[3] J. Kessler, \u201cScattertext: a Browser-Based Tool for Visualizing how Corpora Differ,\u201d in Proceedings of ACL 2017, System Demonstrations, Vancouver, Canada, Jul. 2017, pp. 85\u201390. Accessed: Oct. 19, 2022. [Online]. Available: https:\/\/aclanthology.org\/P17-4015
[4] \u201cspaCy \u00b7 Industrial-strength Natural Language Processing in Python.\u201d https:\/\/spacy.io\/ (accessed Oct. 19, 2022).<\/p>\n","protected":false},"excerpt":{"rendered":"
Gaining insights from text-based data can be a daunting task, even when the data is labeled with ground truth categories and ready for usage in machine learning tasks.Researchers often rely on simple methods like the frequency of words in each category to understand the collection\u2019s characteristics. However, this approach is not always insightful, as term… Continue reading Log Odds Ratio: Going Beyond Simple Term Frequencies to Characterize Textual Categories<\/span><\/a><\/p>\n","protected":false},"author":10,"featured_media":0,"comment_status":"closed","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[21],"tags":[64],"yoast_head":"\nLog Odds Ratio: Going Beyond Simple Term Frequencies to Characterize Textual Categories - Team Acua<\/title>\n<meta name=\"robots\" content=\"index, follow, max-snippet:-1, max-image-preview:large, max-video-preview:-1\" \/>\n<link rel=\"canonical\" href=\"https:\/\/acua.qcri.org\/blog\/log-odds-ratio-going-beyond-simple-term-frequencies-to-characterize-textual-categories\/\" \/>\n<meta property=\"og:locale\" content=\"en_US\" \/>\n<meta property=\"og:type\" content=\"article\" \/>\n<meta property=\"og:title\" content=\"Log Odds Ratio: Going Beyond Simple Term Frequencies to Characterize Textual Categories - Team Acua\" \/>\n<meta property=\"og:description\" content=\"Gaining insights from text-based data can be a daunting task, even when the data is labeled with ground truth categories and ready for usage in machine learning tasks.Researchers often rely on simple methods like the frequency of words in each category to understand the collection\u2019s characteristics. However, this approach is not always insightful, as term… Continue reading Log Odds Ratio: Going Beyond Simple Term Frequencies to Characterize Textual Categories\" \/>\n<meta property=\"og:url\" content=\"https:\/\/acua.qcri.org\/blog\/log-odds-ratio-going-beyond-simple-term-frequencies-to-characterize-textual-categories\/\" \/>\n<meta property=\"og:site_name\" content=\"Team Acua\" \/>\n<meta property=\"article:published_time\" content=\"2022-10-24T11:10:23+00:00\" \/>\n<meta property=\"og:image\" content=\"https:\/\/quecst.qcri.org\/blog\/wp-content\/uploads\/2022\/10\/blog-img-what-customer-insights.jpeg\" \/>\n<meta name=\"author\" content=\"Hind Almerekhi\" \/>\n<meta name=\"twitter:card\" content=\"summary_large_image\" \/>\n<meta name=\"twitter:label1\" content=\"Written by\" \/>\n\t<meta name=\"twitter:data1\" content=\"Hind Almerekhi\" \/>\n\t<meta name=\"twitter:label2\" content=\"Est. reading time\" \/>\n\t<meta name=\"twitter:data2\" content=\"10 minutes\" \/>\n<script type=\"application\/ld+json\" class=\"yoast-schema-graph\">{\"@context\":\"https:\/\/schema.org\",\"@graph\":[{\"@type\":\"Article\",\"@id\":\"https:\/\/acua.qcri.org\/blog\/log-odds-ratio-going-beyond-simple-term-frequencies-to-characterize-textual-categories\/#article\",\"isPartOf\":{\"@id\":\"https:\/\/acua.qcri.org\/blog\/log-odds-ratio-going-beyond-simple-term-frequencies-to-characterize-textual-categories\/\"},\"author\":{\"name\":\"Hind Almerekhi\",\"@id\":\"https:\/\/acua.qcri.org\/blog\/#\/schema\/person\/180d6b2a32636ac94c2fecb1839931cc\"},\"headline\":\"Log Odds Ratio: Going Beyond Simple Term Frequencies to Characterize Textual Categories\",\"datePublished\":\"2022-10-24T11:10:23+00:00\",\"dateModified\":\"2022-10-24T11:10:23+00:00\",\"mainEntityOfPage\":{\"@id\":\"https:\/\/acua.qcri.org\/blog\/log-odds-ratio-going-beyond-simple-term-frequencies-to-characterize-textual-categories\/\"},\"wordCount\":888,\"publisher\":{\"@id\":\"https:\/\/acua.qcri.org\/blog\/#organization\"},\"keywords\":[\"Log Odds Ratio\"],\"articleSection\":[\"Customer segmentation\"],\"inLanguage\":\"en-US\"},{\"@type\":\"WebPage\",\"@id\":\"https:\/\/acua.qcri.org\/blog\/log-odds-ratio-going-beyond-simple-term-frequencies-to-characterize-textual-categories\/\",\"url\":\"https:\/\/acua.qcri.org\/blog\/log-odds-ratio-going-beyond-simple-term-frequencies-to-characterize-textual-categories\/\",\"name\":\"Log Odds Ratio: Going Beyond Simple Term Frequencies to Characterize Textual Categories - Team Acua\",\"isPartOf\":{\"@id\":\"https:\/\/acua.qcri.org\/blog\/#website\"},\"datePublished\":\"2022-10-24T11:10:23+00:00\",\"dateModified\":\"2022-10-24T11:10:23+00:00\",\"breadcrumb\":{\"@id\":\"https:\/\/acua.qcri.org\/blog\/log-odds-ratio-going-beyond-simple-term-frequencies-to-characterize-textual-categories\/#breadcrumb\"},\"inLanguage\":\"en-US\",\"potentialAction\":[{\"@type\":\"ReadAction\",\"target\":[\"https:\/\/acua.qcri.org\/blog\/log-odds-ratio-going-beyond-simple-term-frequencies-to-characterize-textual-categories\/\"]}]},{\"@type\":\"BreadcrumbList\",\"@id\":\"https:\/\/acua.qcri.org\/blog\/log-odds-ratio-going-beyond-simple-term-frequencies-to-characterize-textual-categories\/#breadcrumb\",\"itemListElement\":[{\"@type\":\"ListItem\",\"position\":1,\"name\":\"Home\",\"item\":\"https:\/\/acua.qcri.org\/blog\/\"},{\"@type\":\"ListItem\",\"position\":2,\"name\":\"Log Odds Ratio: Going Beyond Simple Term Frequencies to Characterize Textual Categories\"}]},{\"@type\":\"WebSite\",\"@id\":\"https:\/\/acua.qcri.org\/blog\/#website\",\"url\":\"https:\/\/acua.qcri.org\/blog\/\",\"name\":\"Team Acua\",\"description\":\"Audience, Customer, and User Analytics\",\"publisher\":{\"@id\":\"https:\/\/acua.qcri.org\/blog\/#organization\"},\"potentialAction\":[{\"@type\":\"SearchAction\",\"target\":{\"@type\":\"EntryPoint\",\"urlTemplate\":\"https:\/\/acua.qcri.org\/blog\/?s={search_term_string}\"},\"query-input\":\"required name=search_term_string\"}],\"inLanguage\":\"en-US\"},{\"@type\":\"Organization\",\"@id\":\"https:\/\/acua.qcri.org\/blog\/#organization\",\"name\":\"Team Acua\",\"url\":\"https:\/\/acua.qcri.org\/blog\/\",\"logo\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/acua.qcri.org\/blog\/#\/schema\/logo\/image\/\",\"url\":\"https:\/\/acua.qcri.org\/blog\/wp-content\/uploads\/2022\/10\/cropped-cropped-logo.png\",\"contentUrl\":\"https:\/\/acua.qcri.org\/blog\/wp-content\/uploads\/2022\/10\/cropped-cropped-logo.png\",\"width\":1466,\"height\":770,\"caption\":\"Team Acua\"},\"image\":{\"@id\":\"https:\/\/acua.qcri.org\/blog\/#\/schema\/logo\/image\/\"}},{\"@type\":\"Person\",\"@id\":\"https:\/\/acua.qcri.org\/blog\/#\/schema\/person\/180d6b2a32636ac94c2fecb1839931cc\",\"name\":\"Hind Almerekhi\",\"image\":{\"@type\":\"ImageObject\",\"inLanguage\":\"en-US\",\"@id\":\"https:\/\/acua.qcri.org\/blog\/#\/schema\/person\/image\/\",\"url\":\"https:\/\/secure.gravatar.com\/avatar\/a23aaa08517337c496fdf67313626d2e?s=96&d=mm&r=g\",\"contentUrl\":\"https:\/\/secure.gravatar.com\/avatar\/a23aaa08517337c496fdf67313626d2e?s=96&d=mm&r=g\",\"caption\":\"Hind Almerekhi\"},\"url\":\"https:\/\/acua.qcri.org\/blog\/author\/hialmerekhihbku-edu-qa\/\"}]}<\/script>\n<!-- \/ Yoast SEO plugin. -->","yoast_head_json":{"title":"Log Odds Ratio: Going Beyond Simple Term Frequencies to Characterize Textual Categories - Team Acua","robots":{"index":"index","follow":"follow","max-snippet":"max-snippet:-1","max-image-preview":"max-image-preview:large","max-video-preview":"max-video-preview:-1"},"canonical":"https:\/\/acua.qcri.org\/blog\/log-odds-ratio-going-beyond-simple-term-frequencies-to-characterize-textual-categories\/","og_locale":"en_US","og_type":"article","og_title":"Log Odds Ratio: Going Beyond Simple Term Frequencies to Characterize Textual Categories - Team Acua","og_description":"Gaining insights from text-based data can be a daunting task, even when the data is labeled with ground truth categories and ready for usage in machine learning tasks.Researchers often rely on simple methods like the frequency of words in each category to understand the collection\u2019s characteristics. However, this approach is not always insightful, as term… Continue reading Log Odds Ratio: Going Beyond Simple Term Frequencies to Characterize Textual Categories","og_url":"https:\/\/acua.qcri.org\/blog\/log-odds-ratio-going-beyond-simple-term-frequencies-to-characterize-textual-categories\/","og_site_name":"Team Acua","article_published_time":"2022-10-24T11:10:23+00:00","og_image":[{"url":"https:\/\/quecst.qcri.org\/blog\/wp-content\/uploads\/2022\/10\/blog-img-what-customer-insights.jpeg"}],"author":"Hind Almerekhi","twitter_card":"summary_large_image","twitter_misc":{"Written by":"Hind Almerekhi","Est. reading time":"10 minutes"},"schema":{"@context":"https:\/\/schema.org","@graph":[{"@type":"Article","@id":"https:\/\/acua.qcri.org\/blog\/log-odds-ratio-going-beyond-simple-term-frequencies-to-characterize-textual-categories\/#article","isPartOf":{"@id":"https:\/\/acua.qcri.org\/blog\/log-odds-ratio-going-beyond-simple-term-frequencies-to-characterize-textual-categories\/"},"author":{"name":"Hind Almerekhi","@id":"https:\/\/acua.qcri.org\/blog\/#\/schema\/person\/180d6b2a32636ac94c2fecb1839931cc"},"headline":"Log Odds Ratio: Going Beyond Simple Term Frequencies to Characterize Textual Categories","datePublished":"2022-10-24T11:10:23+00:00","dateModified":"2022-10-24T11:10:23+00:00","mainEntityOfPage":{"@id":"https:\/\/acua.qcri.org\/blog\/log-odds-ratio-going-beyond-simple-term-frequencies-to-characterize-textual-categories\/"},"wordCount":888,"publisher":{"@id":"https:\/\/acua.qcri.org\/blog\/#organization"},"keywords":["Log Odds Ratio"],"articleSection":["Customer segmentation"],"inLanguage":"en-US"},{"@type":"WebPage","@id":"https:\/\/acua.qcri.org\/blog\/log-odds-ratio-going-beyond-simple-term-frequencies-to-characterize-textual-categories\/","url":"https:\/\/acua.qcri.org\/blog\/log-odds-ratio-going-beyond-simple-term-frequencies-to-characterize-textual-categories\/","name":"Log Odds Ratio: Going Beyond Simple Term Frequencies to Characterize Textual Categories - Team Acua","isPartOf":{"@id":"https:\/\/acua.qcri.org\/blog\/#website"},"datePublished":"2022-10-24T11:10:23+00:00","dateModified":"2022-10-24T11:10:23+00:00","breadcrumb":{"@id":"https:\/\/acua.qcri.org\/blog\/log-odds-ratio-going-beyond-simple-term-frequencies-to-characterize-textual-categories\/#breadcrumb"},"inLanguage":"en-US","potentialAction":[{"@type":"ReadAction","target":["https:\/\/acua.qcri.org\/blog\/log-odds-ratio-going-beyond-simple-term-frequencies-to-characterize-textual-categories\/"]}]},{"@type":"BreadcrumbList","@id":"https:\/\/acua.qcri.org\/blog\/log-odds-ratio-going-beyond-simple-term-frequencies-to-characterize-textual-categories\/#breadcrumb","itemListElement":[{"@type":"ListItem","position":1,"name":"Home","item":"https:\/\/acua.qcri.org\/blog\/"},{"@type":"ListItem","position":2,"name":"Log Odds Ratio: Going Beyond Simple Term Frequencies to Characterize Textual Categories"}]},{"@type":"WebSite","@id":"https:\/\/acua.qcri.org\/blog\/#website","url":"https:\/\/acua.qcri.org\/blog\/","name":"Team Acua","description":"Audience, Customer, and User Analytics","publisher":{"@id":"https:\/\/acua.qcri.org\/blog\/#organization"},"potentialAction":[{"@type":"SearchAction","target":{"@type":"EntryPoint","urlTemplate":"https:\/\/acua.qcri.org\/blog\/?s={search_term_string}"},"query-input":"required name=search_term_string"}],"inLanguage":"en-US"},{"@type":"Organization","@id":"https:\/\/acua.qcri.org\/blog\/#organization","name":"Team Acua","url":"https:\/\/acua.qcri.org\/blog\/","logo":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/acua.qcri.org\/blog\/#\/schema\/logo\/image\/","url":"https:\/\/acua.qcri.org\/blog\/wp-content\/uploads\/2022\/10\/cropped-cropped-logo.png","contentUrl":"https:\/\/acua.qcri.org\/blog\/wp-content\/uploads\/2022\/10\/cropped-cropped-logo.png","width":1466,"height":770,"caption":"Team Acua"},"image":{"@id":"https:\/\/acua.qcri.org\/blog\/#\/schema\/logo\/image\/"}},{"@type":"Person","@id":"https:\/\/acua.qcri.org\/blog\/#\/schema\/person\/180d6b2a32636ac94c2fecb1839931cc","name":"Hind Almerekhi","image":{"@type":"ImageObject","inLanguage":"en-US","@id":"https:\/\/acua.qcri.org\/blog\/#\/schema\/person\/image\/","url":"https:\/\/secure.gravatar.com\/avatar\/a23aaa08517337c496fdf67313626d2e?s=96&d=mm&r=g","contentUrl":"https:\/\/secure.gravatar.com\/avatar\/a23aaa08517337c496fdf67313626d2e?s=96&d=mm&r=g","caption":"Hind Almerekhi"},"url":"https:\/\/acua.qcri.org\/blog\/author\/hialmerekhihbku-edu-qa\/"}]}},"jetpack_featured_media_url":"","_links":{"self":[{"href":"https:\/\/acua.qcri.org\/blog\/wp-json\/wp\/v2\/posts\/377"}],"collection":[{"href":"https:\/\/acua.qcri.org\/blog\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/acua.qcri.org\/blog\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/acua.qcri.org\/blog\/wp-json\/wp\/v2\/users\/10"}],"replies":[{"embeddable":true,"href":"https:\/\/acua.qcri.org\/blog\/wp-json\/wp\/v2\/comments?post=377"}],"version-history":[{"count":69,"href":"https:\/\/acua.qcri.org\/blog\/wp-json\/wp\/v2\/posts\/377\/revisions"}],"predecessor-version":[{"id":504,"href":"https:\/\/acua.qcri.org\/blog\/wp-json\/wp\/v2\/posts\/377\/revisions\/504"}],"wp:attachment":[{"href":"https:\/\/acua.qcri.org\/blog\/wp-json\/wp\/v2\/media?parent=377"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/acua.qcri.org\/blog\/wp-json\/wp\/v2\/categories?post=377"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/acua.qcri.org\/blog\/wp-json\/wp\/v2\/tags?post=377"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}