Building a 'search as you type' engine for a Python Flask application using Javascript

In this blog post, I will describe how to build a 'search as you type' feature for a Flask application together with a Javascript frontend. 'Search as you type' means that the search results are displayed while the user is typing into the search field. Google has now included this feature in their search engine.

In my case, this search feature is used to search job advertisements for jobs in academic research and you can see it here in action.

We start with the HTML markup. All we are going to do here is to define a search field, where the user can type something, and a target field, where we are going to show the result:

<fieldset>
     <div class="form-group">
         <div class="col-xs-10 col-sm-8 col-md-6 col-lg-6">
             <input type="text" class="form-control" id="job_query" 
             placeholder="search term" value="" oninput="update_job_table()">
        </div>
    </div>
</fieldset>
<input id="timestamp" value="0" hidden>

<table class="table table-striped">
    <thead>
        <!-- whatever -->
    </thead>
    <tbody id="job_market_table">
    </tbody>
</table>

You can see that if the user types something in the search field the javascript function update_job_table() is called. In this function, we now have to take the current search string, send it to our Flask application, perform a quick search in the database and return the results for this search term.

To send the current search string to Flask we use Javascript. We are going to show a spinner symbol while the user is waiting to get the results back, so I include the spinner.js package. This package has only about 4kb when minimized, so does not add much overhead.

You can set up the spinner by defining the following options:

var opts = {
    lines: 9,    // The number of lines in the spinner
    length: 10,  // The length of each line
    width: 3,    // The line thickness
    radius: 6,   // The radius of the inner circle
    corners: 1,  // Corner roundness (0..1)
    rotate: 58,  // The rotation offset
    direction: 1, // 1: clockwise, -1: counterclockwise
    color: '#000000 ', // #rgb or #rrggbb or array of colors
    speed: 0.9,  // Rounds per second
    trail: 100,  // Afterglow percentage
    shadow: false, // Whether to render a shadow
    hwaccel: false, // Whether to use hardware acceleration
    className: 'spinner', // The CSS class to assign to the spinner
    zIndex: 99,  // the modals have zIndex 100
    top: 'auto', // Top position relative to parent
    left: '50%'  // Left position relative to parent
};

Now we can write the Javascript which sends the search term to the Flask app using Ajax:

function update_job_table() {
    var job_table = document.getElementById('job_market_table');
    job_table.innerHTML = "<td colspan='5'>Updating table</td>";
    var spinner = new Spinner(opts).spin(job_table);

    var search_term = document.getElementById('job_query').value;
    doc = { 'search_term': search_term };
    var timestamp = +new Date();
    doc['timestamp'] = timestamp;
    $.ajax({
        url: $SCRIPT_ROOT + '/search_job_market',
        type: "POST",   
        async: true,
        cache: false,
        dataType: 'json',
        contentType: "application/json; charset=utf-8",
        data: JSON.stringify(doc, null, '\t'),
        success: function(data) {
            if(data.error){
                console.log("Error = ", data.error);
            }
            else{
                var results_timestamp = document.getElementById('timestamp');
                if(data.timestamp > parseInt(results_timestamp.value)){
                    spinner.stop();
                    results_timestamp.value = data.timestamp;
                    job_table.innerHTML = data.html;
                }
            }
        }
    });
}

So we just access the search term through the id and send it with a post request to the Flask /search_job_market view as a JSON object. The $SCRIPT_ROOT variable is set to the home path of the application.

Now we have to handle this request within the /search_job_market'  view.

@app.route('/search_job_market', methods=['POST'])
def search_job_market():
    ''' 
    We get here if the user types into the job market search field
    '''
    if request.method == 'POST': 
        html = get_job_market_table_html(request.json)
        return json.dumps({'success': 'search successful', 'html': html, 
               'timestamp': request.json['timestamp']}),\
               200, {'ContentType':'application/json'}

The function get_job_market_table_html() retrieves the necessary information from the database and renders the HTML. My setup uses an Elasticsearch database, but of course, any database system can be plugged in here. You should probably make sure that your setup is able to return some meaningful result to the user in a reasonable time.

You might have noticed that I pass a timestamp through to all requests. The reason for that is to ensure that we only update the user view with new results. Imagine that the user types 'science' in the search field. This will trigger searches for 's', 'sc', 'sci', 'scie', 'scien', 'scienc' and 'science'. It is possible that the result for 's' takes longer and therefore gets sent back to the page after 'sc' has been displayed. Using the timestamp, our setup just ignores the older result.

I posted the entire code on GitHub. Feel free to leave comments or questions below.
cheers
Florian