Received November 3, 2019, accepted December 13, 2019, date of publication December 25, 2019, date of current version January 6, 2020. _Digital Object Identifier 10.1109/ACCESS.2019.2962420_ ## Entropy of Polysemantic Words for the Same Part of Speech ## 1 2 3 MIHAELA COLHON (10) , FLORENTIN SMARANDACHE © , AND DAN VALERIU VOINEA (iD) 1Department of Computer Science, Faculty of Sciences, University of Craiova, 200585 Craiova, Romania 2Department of Mathematics, The University of New Mexico, Albuquerque, NM 87131, USA 3Faculty of Letters, University of Craiova, 200585 Craiova, Romania Corresponding author: Mihaela Colhon (mcolhon@inf.ucv.ro) **ABSTRACT** In this paper, a special type of polysemantic words, that is, words with multiple meanings for the same part of speech, are analyzed under the name of neutrosophic words. These words represent the most difficult cases for the disambiguation algorithms as they represent the most ambiguous natural language utterances. For approximate their meanings, we developed a semantic representation framework made by means of concepts from neutrosophic theory and entropy measure in which we incorporate sense related data. We show the advantages of the proposed framework in a sentiment classification task. **INDEX TERMS** Neutrosophic sets, semantic word representation, sentiment classification. ## **I. INTRODUCTION** Every natural language word can have multiple realisations from the part-of-speech point of view, and for each of its possible parts-of-speech, it can have multiple meanings (especially the English words). Each sense creates a ‘‘subdimension’’ in the word’s space determined by the partof-speech to which it belongs in the given statement. The polysemantic words (words with multiple senses) can be described by several spaces (one space for each possible part-of-speech) and each space can include several subspaces determined by the meanings the word can have. In this manner, every dimension describes a certain facet of the analysed word. It is also true that certain senses are more frequent than others and in this manner they can force a certain facet to be more prominent than others. We need a comprehensive and unitary study for natural language words formulated as a Multicriteria Decision Making problem [1] in which uncertainty is inevitably involved due to the subjectivity of humans [2]. It has been shown that different senses of the same word usually imply different sentiment orientations for the word under analysis. For instance, the word ‘‘good’’: in ‘‘good man’’ produces a positive utterance while in ‘‘good fight’’ indicates a negative statement. As a direct consequence we need studies that address both the interaction between word sense disambiguation and The associate editor coordinating the review of this manuscript and approving it for publication was Pasquale De Meo. sentiment analysis. These are quite new studies in the literature as the researchers in this area must be intrigued by the usability of sense level information in sentiment analysis. Some researchers take this approach and compute the polarity score for each word sense [3], [4]. The present paper proposes a novel approach for word sentiment classification by extracting a set of semantic data from the SentiWordNet in order to compute a final estimation of the word polarity. SentiWordNet [3], [5] is a well-known freely available lexical resource for sentiment analysis which annotates each sense of a word with three polarity scores. These polarity scores represent the positivity, objectivity and negativity degrees of the annotated word sense ranging from 0 to 1 with their sum up to one. SentiWordNet (SWN) was built on the semantically-oriented WordNet [6], [7], which in its primary form, that is for English language, comprises 155287 words and 117659 senses. There are two main approaches for sentiment analysis: machine learning and knowledge-based. From the machine learning perspective, the Support Vector Machines (SVM) classification (see, for example, [8], [9]) has the best classification performance for sentiment analysis [10], [11] outperforming both the Naïve Bayes and Maximum entropy classification methods. The knowledge-based methods usually make use of the most common sense of the words and in this manner an improvement of accuracy over the baseline was observed [12]. Also, the overall polarities of different senses in each part-of-speech tag categories are also This work is licensed under a Creative Commons Attribution 4.0 License. For more information, see http://creativecommons.org/licenses/by/4.0/ 2975 VOLUME 8, 2020 M. Colhon _et al._ : Entropy of Polysemantic Words for the Same Part of Speech determined [13]. However, the commonly used n-gram features are not robust enough and show widely varying behaviour across different domains [14]. The method we propose in this paper offers a knowledgebased solution for semantic word representation which targets sentiment classification and makes use of the general concepts of neutrosophic theory and entropy measure. A previous study that applies neutrosophic theory in sentiment analysis domain is given in [15]. In this paper we concentrate our approach by keeping in mind only the most difficult cases for sentiment classification. They are represented by a special class of polysemantic words with different meanings for the same part-of-speech realisation. In the present paper these words are named _neutrosophic words_ because their representation involves the core concepts of neutrosophic theory. With this article we are in line with the neutrosphic word representations firstly proposed in [16] and then refined in [17] in which the SentiWordNet (shortly SWN) sentiment scores are interpreted as truth-fullness degrees. The study proposed in this paper also makes use of the SWN polarity scores of each word’s sense, this time in order to determine the overall sentiment score value. The involved computations apply entropy on the words’ sentiment scores as a measure of disorder for the words’ polarities. The paper is organised as follows: the Related Works section overviews the most commonly used multi-space representation techniques in neutrosophy. Section III presents the proposed semantic-level representation which treats the words as union of neutrosophic sets. In Section IV we show how this type of representation can be used in conjunction with a sentiment analysis study. Section V exemplifies all the involved theoretical concepts on a study case also providing the obtained results and the last section is dedicated to the conclusions and our future directions. ## **II. RELATED WORKS** The concept of multi-space was introduced by Smarandache in 1969 [18] by following the idea of hybrid mathematics - especially hybrid geometry [19], [20] for combining different fields into a unifying field [21]–[24]. Let be a universe of discourse and a subset _S_ ⊆ . Let [0 _,_ 1] be a closed interval and three subsets _T , I , F_ ⊆ [0 _,_ 1]. Then, a relationship of an element _x_ ∈ _S_ with respect to the subset _S_ is _x_ ( _T , I , F_ ), which means the following: the _confidence set_ of _x_ is _T_ , the _indefinite set_ of _x_ is _I_ , and the _failing set_ of _x_ is _F_ . A set _S_ , together with the corresponding three subsets _T , I , F_ for each element _x_ in _S_ , is said to be a _neutrosophic set_ [19], [25]. Let be a set and _A_ 1, _A_ 2, _. . ._ , _Ak_ ⊆ . Define 3 _k_ functions _f_ 1 _[z]_[,] _[f]_ 2 _[ z]_[,] _[. . .]_[,] _[f][ z] k_[by] _[f][ z] i_ : _Ai_ → [0 _,_ 1], 1 ≤ _i_ ≤ _k_ , where _z_ ∈{ _T , I , F_ }. If we denote by ( _Ai_ ; _fi[T][,][ f][ I] i[,][ f][ F] i_[)][the][subset] _[A][i]_ together with three functions _fi[T]_[,] _[ f] i[ I]_[,] _[ f] i[ F]_[, 1][ ≤] _[i]_[ ≤] _[k]_[, then [19]:] **==> picture [73 x 31] intentionally omitted <==** is a union of neutrosophic sets which are generalizations of classical sets. Indeed, if we take _fi[T]_ = 1, _fi[I]_[=] _[f] i[ F]_ = 0 for _i_ = 1 _, k_ we obtain [19]: **==> picture [184 x 31] intentionally omitted <==** and correspondingly, for _fi[T]_ = _fi[I]_[=][0,] _[ f][ F] i_ = 1, _i_ = 1 _, k_ we obtain the complementary sets [19]: _k k_ ~~U~~ ( _Ai_ ; _fi[T][,][ f] i[ I][,][ f] i[ F]_[)][ =] U _Ai i_ =1 _i_ =1 The appurtenance and non-appurtenance is obtained if there is an integer _s_ such that _fi[T]_ = 1, _fi[I]_[=] _[ f][ F] i_ = 0, 1 ≤ _i_ ≤ _s_ , but _fj[T]_ = _fj[I]_[=][ 0,] _[ f][ F] j_ = 1, _s_ + 1 ≤ _j_ ≤ _k_ . **==> picture [151 x 32] intentionally omitted <==** The general neutrosophic set is obtained if there is an integer _l_ such that _fl[T]_ ̸= 1 for 1 ≤ _l_ ≤ _s_ , or _fl[F]_ ̸= 1 for _s < l_ ≤ _n_ . The resulted union cannot be represented by abstract sets. ## **III. SEMANTIC-LEVEL REPRESENTATION FOR WORDS** As we have already pointed out in the Introduction section, a word is not a simple data, it can have several (syntactic) attributes and can support more than one semantic interpretations. Metaphorically speaking a word is like a diamond: it can brighter a life or, by contrary, it can cut and destroy. But, from our study point of view, a word is just an entity that can have multiple semantic facets. As we have already pointed out, a word can have more than one part-of-speech, like the word ‘‘good’’ which can be adjective, noun or adverb and to which we dedicate an extensive study in the Section V. There are programs that can automatically identify the part-of-speech of a certain word in a given context. These programs are named Part-Of-Speech Taggers and for most of the languages their accuracy is quite high (more than 90%). On contrary, determining the meaning of a polysemous word in a specific context - that is, performing a disambiguation on the word’s senses, can be a laborious task. In spite of the great number of existing disambiguation algorithms, the problem of word sense disambiguation remains an open one [26]. For some languages like English the accuracy of the disambiguation algorithms does not overcome 75%. It is obviously that we need to model indeterminacy in the semantic word representations. This is the reason why, in the present study we choose to model word representations using neutrosophic theory as, in contrast to intuitionistic fuzzy sets and also interval valued intuitionistic fuzzy sets, indeterminacy degree of an element is explicitly expressed by the neutrosophic sets [27]. Moreover, in [29] the authors 2976 VOLUME 8, 2020 M. Colhon _et al._ : Entropy of Polysemantic Words for the Same Part of Speech state that single valued neutrosophic (SVN) set is a better tool to deal with incomplete, inconsistent and indeterminate information than fuzzy set (FS) and intuitionistic FS (IFS). With the present study we are in line with these assumptions continuing also our previous works in which the natural language words are modelled as single-valued neutrosophic sets in order to approximate their ambiguous meaning [16], [17]. given in Definition 1. We consider the general case in which a word can have _n_ possible parts of speech _POS_ 1 _, . . . , POSn_ , with _n_ ≥ 1, and for each part of speech _POSj_ the word can have _kj_ senses, _kj_ ≥ 1. The representation given in : **==> picture [240 x 67] intentionally omitted <==** In the representation we propose in this paper a word can have multiple dimensions organised on several plans: - the POS plans are determined by the possible part of speech data of the word Using the representation given in Equation 3, the senses corresponding to a certain part of speech _POSj_ with _j_ ∈ {1 _, . . . n_ }, can be obtained as follows: - each POS plan can have several sense units, determined by the possible word’s senses under that POS data - finally, each sense unit is made of some components (sentiment scores) which describe the sense meaning polarity **==> picture [219 x 47] intentionally omitted <==** ## _A. WORDS AS UNION OF NEUTROSOPHIC SETS_ Furthermore, we can apply another filtering on word representation given in Equation 4 if we consider the case in which a specific sense of the word _w_ results to be realised in a given context. Let us note this sense with _sensem_ ; _POSj_ with _m_ ∈{1 _, kj_ }. By applying concepts from neutrosophic sets theory we obtain: The first step in creating a semantic representation is to decide what features to use and how to encode these features. From the features set a word can have, in this study we consider the part-of-speech as the syntactic feature and the word’s sense(s) as its semantic interpretation(s). In what follows, we name semantic facets or simply facets - all the word’s data based on which the semantic interpretation can be defined. Using concepts from neutrosophic sets theory [30] we propose the following semantic representation of a word. **==> picture [243 x 47] intentionally omitted <==** _Definition 1:_ The semantic representation of a word by means neutrosophic theory concepts is defined as: **==> picture [440 x 107] intentionally omitted <==** ## where: - _k_ represents the number of senses the word can have - _fi[T]_[,] _[f][ I] i_[,] _[f] i[ F]_ : _Facets_ → [0 _,_ 1] are the membership functions for the _sensei_ , _i_ = 1 _, k_ , such that: The representation given in Equation 5 corresponds to the most unambiguous case, more precisely to the situation in which we know both the word’s part of speech (noted here with _POSj_ ) and the word sense (noted with _sensem_ ; _POSj_ ). - _fi[T]_ represents the membership degree, - _fi[I]_[represents the indeterminacy degree and] - **–** _fi[F]_ is the degree of nonmembership degree - _Facets_ set includes all the data that characterise the word from the semantic point of view. But, the problems with natural language processing comes from ambiguity - when we could not identify (using automatic tools) which sense is realised in the given context from the set of the word’s possible senses (noted here with ∪ _ki_ = _j_ 1 _[sense][i]_[;] _[POS] j_[). This ambiguity case is depicted by the gen-] eral case given in Equation 3. In this assertion, a word becomes a union of neutrosophic sets. For the _i_ th sense of the word _w_ , the membership functions of the word’s semantic facets fulfil the following properties: **==> picture [207 x 13] intentionally omitted <==** In what follows we will use a simplified form of Equation 3 in which _POSj_ data is removed from the annotations sequences corresponding to the senses and membership functions. Thus, Equation 3 becomes: and if _Facets_ = { _x_ 1 _, . . . , xm_ } then: **==> picture [190 x 31] intentionally omitted <==** **==> picture [191 x 33] intentionally omitted <==** In order to include the information about the part-of-speech data (shortly POS data) we need to refine the representation 2977 VOLUME 8, 2020 M. Colhon _et al._ : Entropy of Polysemantic Words for the Same Part of Speech In the next section we present a method by means of which we can eliminate the ‘‘noises’’ from an ambiguous semantic word representation, more precisely, a representation that includes more than one possible sense. We resolve these issues using Neutrosophic Theory and Entropy measure. Our proposal is described in conjunction with a sentiment analysis study in which the semantic word representation has the form of a three sentiment scores tuple. ## **IV. WORD SEMANTIC REPRESENTATION WITH SENTIMENT SCORES** Sense discrimination addresses words with multiple senses and is done in conjunction with a particular context in which only one sense is realised. This analysis has a semantic nature and is quite difficult to perform it using automatic tools, especially if the realisation context is poor in information that could filter the correct word meaning from the set of possible ones. In order to exemplify our proposal we choose to interpret the word semantics from a sentiment analysis the point of view. Thus, each sense of a word will be represented using its sentiment scores. In what follows, let us consider the approach firstly proposed in [16] and then extended in [17] in which a word _w_ is interpreted as a single-value neutrosophic set constructed upon its sentiment scores which describe the word’s senselevel polarity information being denoted in what follows with ( _sc_ + _, sc_ 0 _, sc_ −), where: - _sc_ + denotes the word positive score, - _sc_ 0 represents the word neutral score and - _sc_ − stands for the word negative score. As in [16] and [17] we use SentiWordNet lexical resource for providing the required information for the sentiment scores of the English words. For a word _w_ with _kj_ senses under a _POSj_ part-of-speech realisation, the semantic representation is defined as a union of the tuples: _sensei_ = ( _sc_ + _i, sc_ 0 _i, sc_ − _i_ ) with _i_ ∈{1 _, . . . , kj_ }. The Equation 6 becomes: **==> picture [211 x 33] intentionally omitted <==** with _sc_ + _i, sc_ 0 _i, sc_ − _i_ ∈ [0 _,_ 1]. The semantic representation given in Equation 7 implies that each word’s sense will include three facets: the positive, the neutral and the negative one. By preserving the notation where + stands for positive, 0 for neutral and − for negative facet, we take _Facets_ = {+ _,_ 0 _,_ −}. The representation given in Equation 7 can be rewritten as: **==> picture [243 x 49] intentionally omitted <==** where _fi[T]_[(] _[x]_[),] _[f] i[ I]_[(] _[x]_[)][and] _[f][ F] i_[(] _[x]_[)][represents][the][membership] functions corresponding to the facet _x_ of the _i_ th sense, **==> picture [242 x 49] intentionally omitted <==** _Remark:_ For the representation given in Equation 8, the default case corresponds to the maximum certainty case where no imprecision occurs which, in terms of membership function is depicted by _fi[T]_[(][{+][|][0][|][−][}] _[i]_[)][=][1,] _[f] i[ I]_[(][{+][|][0][|] −} _i_ ) = 0, _fi[F]_[(][{+ |][ 0][ | −}] _[i]_[)][ =][ 0,] _[ i]_[ =] 1 _, kj_ . We preface the study that addresses the multi-facets words by enumerating the form in which the _one facet words_ are represented in our proposal. These words are the extreme cases of our study and every neutrosophic study provides them. _Case 1:_ If _sc_ + _i_ = 1, _sc_ 0 _i_ = _sc_ − _i_ = 0 and _fi[T]_[(][{+][|][0][|] −} _i_ ) = 1, _fi[I]_[(][{][+][|][0][|][−}] _[i]_[)][=][0,] _[f][ F] i_[(][{+][|][0][|][−}] _[i]_[)][=][0][for] every _i_ = 1 _, kj_ then: **==> picture [231 x 36] intentionally omitted <==** The interpretation of Case 1 is: for all the senses corresponding to the _POSj_ part-of-speech the word _w_ is _pure positive_ . _Case 2:_ If _sc_ + _i_ = _sc_ 0 _i_ = 0, _sc_ − _i_ = 1 and _fi[T]_[(][{+][|][0][|] −} _i_ ) = 1, _fi[I]_[(][{][+][|][0][|][−}] _[i]_[)][=][0,] _[f][ F] i_[(][{+][|][0][|][−}] _[i]_[)][=][0][for] every _i_ = 1 _, kj_ then: **==> picture [231 x 37] intentionally omitted <==** The interpretation of Case 2 is: for all the senses corresponding to the _POSj_ part-of-speech the word _w_ is _pure negative_ . _Case 3:_ If _sc_ + _i_ = _sc_ − _i_ = 0, _sc_ 0 _i_ = 1 and _fi[T]_[(][{+][|][0][|] −} _i_ ) = 1, _fi[I]_[(][{][+][|][0][|][−}] _[i]_[)][=][0,] _[f][ F] i_[(][{+][|][0][|][−}] _[i]_[)][=][0][for] every _i_ = 1 _, kj_ then: **==> picture [231 x 37] intentionally omitted <==** The interpretation of Case 3 is: for all the senses corresponding to the _POSj_ part-of-speech the word _w_ is _pure neutral_ . These three cases correspond to the non-ambiguous words, that is, words with a unique sense (one semantic representation) or similar semantic representations for all of their possible senses. Since in a natural language there are many words (especially in English) with multiple senses - the _polysemantic words_ , in what follows we will concentrate our study only on these words. For the polysemantic words we get different semantic representations that must be resolved by dealing with many degrees of uncertainties. In this case, the simple reunion of their semantic dimensions is a general neutrosophic set that cannot be formalised using abstract set theories. For this reason, in the next definition we introduce the concept of _neutrosophic word_ in conjunction with a sentiment analysis. 2978 VOLUME 8, 2020 M. Colhon _et al._ : Entropy of Polysemantic Words for the Same Part of Speech _Definition 2:_ A _neutrosophic word_ is a polysemantic word that under the same part of speech realization has at least two different sentiment polarities which means: (∃( _w_ ) _POSj_ with _kj >_ 1 senses) ∧ (∃ _i_ 1 _, i_ 2 ∈{1 _, . . . , kj_ } _, i_ 1 ̸= _i_ 2: _sensei_ 1 ̸= _sensei_ 2) Different sense tuples imply different sentiment scores and we obtain: (∃( _w_ ) _POSj_ with _kj >_ 1 senses) ∧ [∃ _i_ 1 _, i_ 2 ∈{1 _, . . . , kj_ } _, i_ 1 ̸= _i_ 2: ( _sc_ + _i_ 1 _, sc_ 0 _i_ 1 _, sc_ − _i_ 1 ) ̸= ( _sc_ + _i_ 2 _, sc_ 0 _i_ 2 _, sc_ − _i_ 2 )] As a direct consequence, the semantic representation of neutrosophic words is: **==> picture [221 x 24] intentionally omitted <==** **==> picture [138 x 13] intentionally omitted <==** with _sc_ + _i_ 1 ̸= _sc_ + _i_ 2 or _sc_ 0 _i_ 1 ̸= _sc_ 0 _i_ 2 or _sc_ − _i_ 1 ̸= _sc_ − _i_ 2 and _fi[T]_ 1[(][{+][|][0][|][−}][)] _[,][ f][ T] i_ 2[(][{+][|][0][|][−}][)] _[>]_[0,] _[ i]_[1][̸=] _[i]_[2][. By the fact] that the membership degrees are greater than 0, we obtain for a neutrosophic word _w the necessity of having (at least) two different sentiment representations_ for the same ( _w_ ) _POSj_ . The neutrosophic theory means from the very beginning dealing with uncertainty. This is also true for the neutrosophic words. These words can be evidenced in case of an imprecise disambiguation mechanism which fails in recognising what sense is realised in the given context even if the part-of-speech data is correctly provided. In our approach, a neutrosophic word is synonym with a word that has different sense facets and for which we cannot establish a unique semantic representation. For the chosen sentiment analysis exemplification, different sense facets for a word means different sentiment scores tuples. In the next section we exemplify how the proposed method works. We show that using the neutrosophic sets theory and applying the entropy measure on the word representations we can identify the word’s sentiment facet with respect to the given part-of-speech. ## _A. ENTROPY AS A MEASURE OF UNCERTAINTY FOR THE NEUTROSOPHIC WORDS REPRESENTATIONS_ Fuzzy entropy, distance measure and similarity measure are three basic concepts used in fuzzy sets theory [27]. Among them, Entropy is an efficient tool to model uncertainty [28] or, in layman terms, Entropy is a measure of disorder. It can be used in order to measure how disorganised an input values set is by calculating the entropy of their values/labels. Entropy is high if the input values are highly varied and low if many input data have the same value. In mathematical terms, Entropy is defined as the sum of the probability of each input values or labels times the log probability of that label: **==> picture [193 x 20] intentionally omitted <==** where _P_ ( _l_ ) is the frequency probability of the _label_ item in the considered data and _labels_ denotes the set of possible labels. From this definition we obtain that labels with low frequency do not affect much the entropy (because _P_ ( _l_ ) is small). The same result for labels with high frequency as in their case, _log_ 2 _P_ ( _l_ ) is small. Only when the inputs have wide varieties of labels (and as a direct consequence, these many labels have a medium frequency) the entropy is high because neither _P_ ( _l_ ) nor _log_ 2 _P_ ( _l_ ) is small. Entropy has values between 0 and 1 and high entropy values stand for high levels of disorder or ‘‘low level of purity’’. Following this property, we can qualify the uncertainty of the words’ semantic nature by applying the entropy measure on their sense representation labels: _the higher the values for entropy measure the higher the level of uncertainty for the analyzed word representations_ . The neutrosophic word is a concept with more than one possible sense for at least one of its possible part-of-speech data. On the other hand, entropy is a measure of uncertainty. Between the possible senses we can have certain similarity degrees and the entropy measure can be used in order to determine how similar or dissimilar these senses are. The most common manner to unify a set of possible representations into a single one is to consider only the maximum (or the minimum) value or to average the values (in our case, the sentiment scores) as in the following formula: **==> picture [240 x 75] intentionally omitted <==** where _kj_ notes the number of senses for the analysed word. But this method of unifying different representation can be trustful only if the averaged values are not very dissimilar with the initial ones. _Example 1:_ Let us consider a word _w_ with two extreme sentiment scores tuples: (0 _,_ 0 _,_ 1) and (1 _,_ 0 _,_ 0). Overall, we obtain two different facets: in the first representation we have a _pure positive word_ while in the second we get a _pure negative word_ . If we merge these two representation by averaging their sentiment scores values we get (0 _._ 5 _,_ 0 _,_ 0 _._ 5) - a representation that could be interpreted as a _neutral word_ . Definitely this would be a wrong classification for a strong sentiment word. We define a bijective mapping for labelling the sentiment score values to a set of three strength degrees, _SD_ = { _low, medium, high_ }. We obtain _sd_ : [0 _,_ 1] → _SD_ with: **==> picture [180 x 43] intentionally omitted <==** Mapping the score values to the _SD_ labels we get ‘‘low’’ label for a small score, ‘‘medium’’ for not a small but also not a high score and ‘‘high’’ for a big score. Using these strength degrees we can qualify by means of the entropy measure calculated as in Equation 9 how disorganised the scores values are from the point of view of the sentiment strength. All the involved operations are given in Algorithm 1. 2979 VOLUME 8, 2020 M. Colhon _et al._ : Entropy of Polysemantic Words for the Same Part of Speech The representation given in Equation 12 tells more about _what the word is not_ than about the type the word _is_ as we consider **Example 1** only for showing why the simple unification of multiple representations by averaging their values is not always enough. As one can observe, the representation given in Equation 12 tells with maximum certainty that the word is not a neutral word. For the obtained positive and negative scores the indeterminacy membership functions have maximum values, illustrating in this way a maximum indeterminacy degree. This extreme case is quite rarely, being presented only for its theoretical purpose. **Algorithm 1** Merging Multiple Semantic Representations of a Neutrosophic Word ( _w_ ) _POSj_ **==> picture [186 x 104] intentionally omitted <==** In the next section we apply the proposed method on a real data: a neutrosophic word in its all possible parts of speech. With this complex case we show that the method described in this article succeeds in merging multiple and diverse semantic word representations. OUTPUT: ∪ _x_ ∈ _FacetsAvg_ ( _x_ ) _, f[T]_ ( _x_ ) _, f[I]_ ( _x_ ) _, f[F]_ ( _x_ ) We can now give the manner in which the multiple representations of a neutrosophic word ( _w_ ) _POSj_ can be unified into a unique sentiment representation _Avg_ ( _w_ ) _POSj_ based on the values provided by Algorithm 1: ## **V. STUDY CASE** The word ‘‘good’’ appears in WordNet with three different parts of speech (noun, adjective, and adverb) and with many senses for each of its syntactic labels. We consider this word represents a perfect example for the neutrosophic word concept introduced in this paper and for this reason we dedicate the study case to it. _Avg_ (( _w_ ) _POSj_ ) **==> picture [221 x 72] intentionally omitted <==** In Table 1 are given all the senses the word ‘‘good’’ can have, grouped upon the part-of-speech data. Each sense is given together with the sentiment scores extracted from SentiWordNet and also with its definition and some examples (as they are given in WordNet). **==> picture [18 x 9] intentionally omitted <==** In Algorithm 1 we model the degrees of trustfulness for the resulted average scores representation by means of the membership functions, such that ∀ _x_ ∈ _Facets_ : In Table 2 we gather all the data extracted from SentiWordNet: the word’s parts of speech, the three facets given by the corresponding sentiment scores and the distributions among the senses of the sentiment scores. We also give the entropy measures for each word’s facet in all the three parts of speech and also the average values of the sentiment scores. - If the entropy _Entropy_ ( _x_ ) is small (the minimum value is 0) then the average value _Avg_ ( _x_ ) can approximate with high degree of certainty the initial word’s sentiment scores; in this case the membership function for the facet _x_ is set to a big value (almost 1) as _f[T]_ ( _x_ ) ← 1 − _Entropy_ ( _x_ ). By applying Algorithm 1 on the SentiWordNet scores of the word ‘‘good’’ we obtain the following representations (see also Table 2): - If the entropy _Entropy_ ( _x_ ) is high (the maximum value is 1) then the membership function is set to a small value (almost 0) while the indeterminacy degree _f[I]_ ( _x_ ) is set to be equal with the entropy function value. **==> picture [65 x 10] intentionally omitted <==** **==> picture [476 x 194] intentionally omitted <==** - For preserving the sum of the membership functions to value 1 (see Equation 1), the nonmembership degree _f[F]_ ( _x_ ) for the facet _x_ is always 0. For the case given in **Example 1** we obtain that the entropy corresponding to the positive and negative scores is equal to its maximum value: _E_ (+) = _E_ (−) = 1, while the entropy for the neutral scores is zero. The resulted average representation can be written as follows: 2980 VOLUME 8, 2020 M. Colhon _et al._ : Entropy of Polysemantic Words for the Same Part of Speech **TABLE 1.** The SentiWordNet data for the word ‘‘good’’. **TABLE 2.** The semantic representations of the word ‘‘good’’. The negative scores, being not representative (the greatest value is 0.12), are omitted in the listing. the positive and the neutral. From the results obtained in Equations 13 and 15 we can conclude: - the word ‘‘good’’ as adverb is a neutral word because its neutral average score is 0 _._ 81 with _fADV[T]_[(0)][=][0] _[.]_[59,] a value that exceeds by far its positive average score (0 _._ 18 with _fADV[T]_[(][+][)][ =][ 0] _[.]_[59)] - the word ‘‘good’’ as adjective is a positive word because its positive average score is 0 _._ 61 with _fADJ[T]_[(][+][)] = 0 _._ 59 while the neutral average score is only 0 _._ 38, with _fADJ[T]_[(0)][ =][ 0] _[.]_[59] As a noun, we can consider it positive or neutral word, in both cases with high indeterminate degrees: _fNOUN[I]_[(][+][)][=] _fNOUN[I]_[(0)][=][0] _[.]_[75,][its][average][positive][and][neutral][scores] equal with 0 _._ 5 (see Equation 14). This is the case when additional filters taken from the context in which the word occurs must be applied in order to establish the word semantic facet. ## **VI. CONCLUSION AND FUTURE WORK** As pointed out in [31] each object has a corresponding (fuzzy, intuitionistic fuzzy, or neutrosophic) degree of appurtenance to a set of classification classes, with respect to its attributes’ values. **==> picture [239 x 105] intentionally omitted <==** These results can be interpreted as follows: no matter its part of speech realisation, we can precisely say that the word ‘‘good’’ is NOT a negative word. Two possible facets remain: In the present paper we propose a method that determines the appurtenance degrees of the semantic facets of a natural language word based on the entropy measure. We apply the proposed method on a real data: a polysemantic word in its all possible parts of speech. We prove with this complex study case that the method succeeds in merging multiple and diverse semantic word representations by filtering the ‘‘noises’’ through the entropy function values. The proposed method can be improved in case of high entropy values when additional filters must be applied by taken into account the word contextual data. The developing of these additional filters represents the trigger of our future studies. ## **REFERENCES** > [1] F. Xiao, ‘‘A multiple–criteria decision–making method based on D numbers and belief entropy,’’ _Int. J. Fuzzy Syst._ , vol. 21, no. 4, pp. 1144–1153, Jun. 2019. > [2] F. Xiao, ‘‘EFMCDM: Evidential fuzzy multicriteria decision making based on belief entropy,’’ _IEEE Trans. Fuzzy Syst._ , to be published. 2981 VOLUME 8, 2020 M. Colhon _et al._ : Entropy of Polysemantic Words for the Same Part of Speech - [3] A. Esuli and F. Sebastiani, ‘‘SentiWordNet: A publicly available lexical resource for opinion mining,’’ in _Proc. LREC_ , Genoa, Italy, 2006, pp. 417–422. - [4] J. Wiebe and R. Mihalcea, ‘‘Word sense and subjectivity,’’ in _Proc. COLING/ACL_ , Sydney, Australia, 2006, pp. 1065–1072. - [30] F. Smarandache, ‘‘Neutrosophic set—A generalization of the intuitionistic fuzzy set,’’ in _Proc. 2006 IEEE Int. Conf. Granular Comput._ , Atlanta, GA, USA, Jun. 2006, pp. 38–42. - [31] F. Smarandache, _Plithogeny, Plithogenic Set, Logic, Probability, and Statistics_ . Brussels, Belgium: Pons Publishing House, 2017. - [5] S. Baccianella, A. Esuli, and F. Sebastiani, ‘‘SentiWordNet 3.0: An enhanced lexical resource for sentiment analysis and opinion mining,’’ in _Proc. LREC_ , Valletta, Malta, 2010, pp. 2200–2204. - [6] G. A. Miller, ‘‘WordNet: A lexical database for English,’’ _Commun. ACM_ , vol. 38, no. 11, pp. 39–41, 1995. - [7] C. Fellbaum, _WordNet: An Electronic Lexical Database_ Cambridge, MA, USA: MIT Press, 1998. - [8] S. Mohammad, S. Kiritchenko, and X. Zhu, ‘‘Nrc-canada: Building the state-of-the-art in sentiment analysis of tweets,’’ in _Proc. SemEval_ , Atlanta, GA, USA, 2013, pp. 321–327. - [9] M. C. Mihăescu, ‘‘Classification of users by using support vector machines,’’ in _Proc. WIM_ , Craiova, Romania, 2012, Art. no. 68. - [10] B. Pang and L. Lee, ‘‘Opinion mining and sentiment analysis,’’ _Found. Trends Inf. Ret._ , vol. 2, nos. 1–2, pp. 1–135, 2008. - [11] K. Dave, S. Lawrence, and D. Pennock, ‘‘Mining the peanut gallery: Opinion extraction and semantic classification of product reviews,’’ in _Proc. WWW_ , Budapest, Hungary, 2003, pp. 519–528. - [12] E. Cambria, B. Schuller, B. Liu, H. Wang, and C. Havasi, ‘‘Knowledge– based approaches to concept–level sentiment analysis,’’ _IEEE Intell. Syst._ , vol. 28, no. 2, pp. 12–14, Mar. 2013. MIHAELA COLHON received the Ph.D. degree in computer science from the Department of Computer Science, Faculty of Mathematics and Computer Science, The University of Piteşti, Romania, in 2009. She is currently an Associate Professor with the Department of Computer Science, University of Craiova, Romania. Her research fields are artificial intelligence, with specialization in knowledge representation, natural language processing (NLP), human language technologies (HLT), and computational statistics and data analysis with applications in NLP. - [13] C. Sumanth and D. Inkpen, ‘‘How much does word sense disambiguation help in sentiment analysis of micropost data?’’ in _Proc. 6th Workshop Comput. Approaches Subjectivity, Sentiment Social Media Anal._ , Berlin, Germany, 2015, pp. 115–121. - [14] Z. Fei, J. Liu, and G. Wu, ‘‘Sentiment classification using phrase patterns,’’ in _Proc. IEEE Int. Conf. Comput. Inf. Technol._ Washington, DC, USA, Sep. 2004, pp. 1147–1152. - [15] F. Smarandache, M. Teodorescu, and D. Gîfu, ‘‘Neutrosophy, a sentiment analysis model,’’ in _Proc. RUMOUR_ , Toronto, ON, Canada, 2017, pp. 38–41. - [16] M. Colhon, Ş. Vlˇaduţescu, and X. Negrea, ‘‘How objective a neutral word is? A neutrosophic approach for the objectivity degrees of neutral words,’’ _Symmetry_ , vol. 9, no. 11, p. 280, Nov. 2017. - [17] F. Smarandache, M. Colhon, Ş. Vlˇaduţescu, and X. Negrea, ‘‘Wordlevel neutrosophic sentiment similarity,’’ _Appl. Soft Comput._ , vol. 80, pp. 167–176, Jul. 2019. - [18] F. Smarandache, ‘‘Neutrosophic transdisciplinarity—Multi-space & multistructure,’’ in _Proc. State Archives_ , Valcea Branch, Romania, 1969. - [19] L. Mao, _Smarandache Geometries & Map Theory with Applications (I)_ , (English-Chinese Bilingual Edition). Beijing, China: Chinese Academy of Sciences, 2006. - [20] L. Mao, _Automorphism Groups of Maps, Surfaces and Smarandache Geometries_ . Beijing, China: Beijing Institute of Civil Engineering and Architecture, 2011. - [21] L. Mao, _Smarandache Multi-Space Theory (I)—Algebraic Multi-Spaces_ . New York, NY, USA: Cornell Univ., 2006. - [22] L. Mao, _Smarandache Multi-Space Theory (II)—Multi-Spaces on Graphs_ . New York, NY, USA: Cornell Univ., 2006. - [23] L. Mao, _Smarandache Multi-Space Theory (III)—Map Geometries and Pseudo-Plane Geometries_ . New York, NY, USA: Cornell Univ., 2006. FLORENTIN SMARANDACHE received the M.Sc. degree in mathematics and computer science from the University of Craiova, Romania, and the Ph.D. degree in mathematics from the State University of Kishinev. He held a postdoctoral position in applied mathematics at the Okayama University of Sciences, Japan. He is currently a Professor of mathematics with the University of New Mexico, USA. He has been the Founder of neutrosophy (generalization of dialectics), neutrosophic set, logic, probability, and statistics, since 1995. He has published hundreds of articles on neutrosophic physics, superluminal and instantaneous physics, unmatter, absolute theory of relativity, redshift and blueshift due to the medium gradient and refraction index besides the Doppler effect, paradoxism, outerart, neutrosophy as a new branch of philosophy, Law of Included Multiple-Middle, multispace and multistructure, degree of dependence and independence between neutrosophic components, refined neutrosophic set, neutrosophic over-under-off-set, plithogenic set, neutrosophic triplet and duplet structures, quadruple neutrosophic structures, DSmT, and so on to many peer-reviewed international journals and many books. He presented articles and plenary lectures to many international conferences around the world. [http://fs.unm.edu/FlorentinSmarandache.htm] - [24] L. Mao, _Proceedings of the First International Conference On Smarandache Multispace & Multistructures_ . Beijing, China: The Educational Publisher Inc., 2013. - [25] F. Smarandache, ‘‘A unifying field in logics: Neutrosophic logic,’’ _Multiple Valued Logic Int. J._ , vol. 8, no. 3, pp. 385–438, 2002. - [26] F. Hristea, _The Naïve Bayes Model for Unsupervised Word Sense Disambiguation. Aspects Concerning Feature Selection_ . Amsterdam, The Netherlands: Springer, 2012. - [27] R. Sahin and M. Karabacak, ‘‘A multi attribute decision making method based on inclusion measure for interval neutrosophic sets,’’ _Int. J. Eng. Appl. Sci._ , vol. 2, pp. 13–15, 1995. - [28] F. Xiao, ‘‘Multi-sensor data fusion based on the belief divergence measure of evidences and the belief entropy,’’ _Inf. Fusion_ , vol. 46, pp. 23–32, Mar. 2019. - [29] P. Liu, Q. Khan, and T. Mahmood, ‘‘Some single-valued neutrosophic power muirhead mean operators and their application to group decision making,’’ _J. Intell. Fuzzy Syst._ , vol. 37, no. 2, pp. 2515–2537, Sep. 2019. DAN VALERIU VOINEA graduated from the Faculty of Economic Sciences and Business Administration, University of Craiova, Romania, the Faculty of History, Philosophy and Geography, University of Craiova, and the Ph.D. degree in sociology from the University of Craiova. He is currently a Senior Lecturer with the University of Craiova. He is also the author or coauthor of more than 30 scientific articles, including Web of science articles and of international seminars and conferences papers. He is a member of CCSCMOP‘s Scientific Committee. 2982 VOLUME 8, 2020