NLP and the information service

How does Natural Language Processing improve my search results?

March 2, 2017
Clare Brown

There is currently a lot of hype and discussion in professional services surrounding the topics of artificial intelligence (AI) and natural language processing (NLP). These are concepts that library and information professionals should be aware of, but what do we really mean when we talk about AI and NLP? And do they have a place within your library and information service?

Simply put, natural language processing can be defined as the ability of a computer programme to understand human speech, i.e. to process natural language. As such, natural language processing is just one of the many branches of artificial intelligence.

Instead of communicating with computer programmes in a technical commanding language, NLP enables a more natural form of communication. Well-known everyday examples of NLP can be found in Apple’s Siri or Amazon’s Alexa - both tools can respond to and understand natural human speech.

Human and machine: How NLP supports and enhances library research

What use does natural language processing have in information services? There are many different ways to incorporate NLP into your library, primarily in support of your search function. In this article, we shall explore four of the ways NLP can be used - through sentiment analysis, entity extraction, keyword searching and concept extraction.

Find out how Vable Inbox made a real difference to this US firm  →

NLP and Boolean Searching

NLP adds a whole new dynamic to traditional Boolean searching. Instead of interpreting the literal meaning of each word, NLP takes the wider context and intent of the word or term into account. This means that it is easy for you to surface an incredibly specific set of search results, containing only the most relevant information and your search becomes more efficient.

What’s more, there is also now the scope to understand the overarching meaning of articles being searched, as opposed to solely the meaning of the words within.

“A computer can know the definition of a word, but it doesn’t understand the meaning of words within a larger context” (Vasco Pedro, 2016)

In this regard, natural language processing is able to interpret the overarching mood of an article. This wouldn’t be possible without the use of NLP (unless done manually of course), since the system is now able to go beyond viewing the article as a compilation of individual words and can instead consider the article as a whole.

Such a process is known as sentiment analysis. Using sentiment analysis would enable you to assess whether articles written about your organisation, clients or competitors are positive or negative in their coverage. You can monitor the overall perception of your organisation or entity of interest in the news.

This could be of particular use for the marketing department who need to have a clear understanding of their organisation’s image. Or it could be to monitor mentions of your clients which would be fed back to your fee-earners. They can then be alerted to reputation, litigation matters or anything else that could affect their public image.

Learn more about searching ›

NLP and Entity Extraction 

A second manifestation of natural language processing in your library can be found in what is known as entity extraction. Since NLP interprets the context and article as a whole, it is able to tag each piece with specific entities, such as the geographic location e.g. United Kingdom or London.

This is far more advanced than just using such a term in your Boolean search criteria as, instead of searching for mentions of “United Kingdom”, the system searches only for articles that are actually about the United Kingdom and so a narrower set of much more relevant results will be surfaced.

Keywords and NLP

This ties in nicely with keyword tagging and searching, whereby NLP is able to go beyond mere mentions of a term in an article and instead consider whether that article is actually substantially focussing on, let’s say, Apple or indeed if it only mentions them in passing. Using such keyword searching also enables you to filter out a great deal of the white noise and avoid bringing up results that mention, in this case, apple the fruit rather than Apple the company.  

NLP and Concept Extraction

Fourthly, natural language processing can categorise concepts into themes for you, even if they are not mentioned directly in the text itself. An example of such concept extraction could be found in searching for articles relating to a specific industry.

Using NLP, the system is able to recognise brands and company names contained in the article text and place these into the appropriate industry. You are then able to search for that specific industry, and all of these relevant articles will come up in your results even if the industry itself isn’t directly mentioned.

New call-to-action