Comparison with PostgreSQL

Postgresql Full-text Search

In order to perform Full-text Search in Postgresql

you need to select a text(in our case it is words_eng.txt)

and place it to postgresql database.

1. Create table

CREATE TABLE words (
    word TEXT
);

2. Use the COPY command to import the data from the file into the "words" table. Assuming the words_eng.txt file is in the same directory as your current location

COPY words FROM 'words_eng.txt';

3. To perform searching operation run this command

SELECT * FROM documents WHERE to_tsvector('english', content) @@ to_tsquery('english', 'search_term');

We decided to connect Postgresql Full-text Search with Flask

and make simple client side to perform full-text search operation

1. main.py code

Import the necessary modules:

from flask import Flask, render_template, request
from flask_sqlalchemy import SQLAlchemy

Initialize your Flask application and configure the database connection:

app = Flask(__name__, template_folder=".")
app.config['SQLALCHEMY_DATABASE_URI'] = 'postgresql://username:password@localhost:5432/database_name'
db = SQLAlchemy(app)

Define a model for your table:

class Word(db.Model):
    __tablename__ = 'words'
    word = db.Column(db.String, primary_key=True)
    content_tsvector = db.Column(db.String)

Implement the full-text search functionality in your Flask route:

@app.route('/search', methods=['GET', 'POST'])
def search():
    if request.method == 'POST':
        search_query = request.form['query']
        results = Word.query.filter(Word.content_tsvector.match(search_query)).all()
        return render_template('search_results.html', results=results)
    return render_template('search.html')

if __name__ == '__main__':
    app.run(debug=True)

2. search.html code

Create the search.html template with a search form:

<form method="POST" action="/search">
    <input type="text" name="query" placeholder="Enter search query">
    <button type="submit">Search</button>
</form>

Create the search_results.html template to display the search results:

3. search_results.html code

{% for result in results %}
    <p>{{ result.word }}</p>
{% endfor %}

Comparison of posgresql full-text search and self written full-text search based on the suffix trie

PostgreSQL Full Text Search:

• Uses a specialized full-text search engine built into the PostgreSQL database.
• Provides advanced features like stemming, ranking, and search operators.
• Supports various languages and allows customization of text processing and search behavior.
• Can handle large amounts of text efficiently and has better performance.
• Makes it easy to combine full-text search with other database operations.

Self-Written Suffix Trie-based Full Text Search:

• Based on a custom implementation of a suffix trie data structure.
• Provides a basic and simplified approach to full-text search.
• Offers a simple and lightweight solution for searching within a limited set of words or small texts.
• Does not require an external full-text search engine or database.
• Can be suitable for scenarios where you have a specific and small set of words or limited search requirements.