Research Publications Authored by SLIIT Staff
Permanent URI for this communityhttps://rda.sliit.lk/handle/123456789/4195
This collection includes all SLIIT staff publications presented at external conferences and published in external journals. The materials are organized by faculty to facilitate easy retrieval.
Browse
3 results
Search Results
Publication Embargo Dynamic stopword removal for Sinhala Language(IEEE, 2019-10-08) Jayaweera, A. A. V. A; Senanayake, Y. N; Haddela, P. SIn the modern era of information retrieval, text summarization, text analytics, extraction of redundant (noise) words that contain a little information with low or no semantic meaning must be filtered out. Such words are known as stopwords. There are more than 40 languages which have identified their language specific stopwords. Most researchers use various techniques to identify their language specific stopword lists. But most of them try to define a magical cut-off point to the list, which they identify without any proof. In this research, the focus is to prove that the cut-off point depends on the source data and the machine learning algorithm, which will be proved by using Newton's iteration method of root finding algorithm. To achieve this, the research focuses on creating a stopword list for Sinhala language using the term frequency-based method by processing more than 90000 Sinhala documents. This paper presents the results received and new datasets prepared for text preprocessing.Publication Embargo Enhanced Tokenizer for Sinhala Language(IEEE, 2019-10-08) Senanayake, S. Y; Kariyawasam, K. T. P. M; Haddela, P. STokenization process plays a prominent role in natural language processing (NLP) applications. It chops the content into the smallest meaningful units. However, there is a limited number of tokenization approaches for Sinhala language. Standard analyzer in apache software library and natural language toolkit (NLTK) are the main existing approaches to tokenize Sinhala language content. Since these are language independent, there are some limitations when it applies to Sinhala. Our proposed Sinhala tokenizer is mainly focusing on punctuation-based tokenization. It precisely tokenizes the content by identifying the use case of punctuation mark. In our research, we have proved that our punctuation-based tokenization approach outperforms the word tokenization in existing approaches.Publication Embargo A rule based stemmer for Sinhala language(IEEE, 2020-04-13) Kariyawasam, K. T. P. M; Wickramasinghe, S. Y; Haddela, P. SStemming, as its word implies it converts the original word into its root/base format which is called as stem. Stemming process plays a prominent role in natural language processing (NLP) because it makes applications more efficient and effective. Though stemming is such an important task, it is hard to find a stemming method for Sinhalese language which is official language of Sri Lanka. There are common language analyzers which cannot be use for stemming since they are highly language dependent. In this paper, we present a rule-based stemming method by using suffix and prefix rules in Sinhalese language.