BRAC University Institutional Repository

Similarity search for Bangla

DSpace/Manakin Repository

Show simple item record Morshed, Mahbub Shahed, Md. Shahid 2011-12-07T10:55:26Z 2011-12-07T10:55:26Z 2011-04
dc.identifier.other ID 09201023
dc.identifier.other ID 07101007
dc.description This thesis is submitted in a partial fulfillment of the requirement for the degree of Bachelor of Science in Computer Science and Engineering en_US
dc.description.abstract Due to typos and misspelling search engines cannot provide users with proper information. Large search engines like Google provides suggestion tab "did you mean". But such options are not included in most of the open source search engines. Our goal was to find a way to implement an exhaustive similarity search in an efficient way and develop such option for Bangla search engine . We used Solr for that. And configured Solr with Lavenstine distance and Jaro Winkler algorithm to provide "Did you mean" for English. But to implement this for Bangla we needed a Stemmer for Bangla and that was not present in SoIr. In order to build a efficient stemmer we need to tag the tokens properly according to their parts of speech as the stemming process for different parts of speech is different. There are different approaches to the problem of assigning a part of speech (POS) tag to each word of a natural language sentence. We have used NLTK toolkit to develop a Regular expression tagger for Bangla verbs using the common suffixes( 1 i r ) found in Bangla grammar. Then we analyzed its performance on main verbs extracted from a 100K token 51 Page tagged-corpus. In this thesis we also compare the performance of a few POS tagging techniques for Bangla language, e.g. statistical approach (ngram) and transformation based approach (Brill's tagger). A supervised POS tagging approach requires a large amount of annotated training corpus to tag properly. We used the 100K token hand tagged corpus developed by Microsoft India to implement these techniques. en_US
dc.language.iso en en_US
dc.publisher Department of Computer Science and Engineering en_US
dc.title Similarity search for Bangla en_US
dc.type Thesis en_US

Files in this item

Files Size Format View
Morshed-09201023, Shahed-07101007.PDF 5.385Mb PDF View/Open or Preview

This item appears in the following Collection(s)

Show simple item record

Policy Guidelines

Search DSpace


My Account