subscribe to our newsletter sign up
Making sense of big data

Making sense of big data

Promoted by

The accelerating volume of information available to today’s legal professionals is outpacing the ability to manage it.



WITH A significant focus on increasing productivity and delivering legal services  more efficiently, there is no doubt that advanced online research platforms using cutting-edge technology play an integral part in improving legal professionals’ efficiency – but how exactly do they work?


How search technology makes sense of data


Search platforms first need to: have access to the largest possible pool of information necessary for legal professionals to carry out comprehensive research; link all of these resources together to help surface the hidden connections and speed up the journey along

the research path. In order for these resources to be linked together correctly and effectively, they must be stored and processed together.

This is the point where the challenges of ‘Big Data’ come into play. The content repository contains millions of documents across many content types which are structured differently. Most of the documents have multiple relationships with other documents, so almost any document in the collection could be affected by any new document which is added. As they are all interlinked, you begin to have a very complicated web of content that requires a parallel data processing and analytics framework to manage the relationships between documents.

This should preferably be open source, and implemented on commodity computing clusters, in order to keep costs down. A distributed data processing framework allows for rapid, high-volume data enrichment. It is used to constantly manage relationships and linkages between documents. In the context of legal information, this includes entity resolution, topic-based document classification, relationship recognition, calculation of document activity scores, and generation of alerts based on user-defined topics or queries.


Algorithms behind the scenes of your search


Now let’s have a look at how search algorithms deliver a comprehensive set of results, with the most relevant documents clearly surfaced. Basic search engine strategies are employed: term frequency-inverse document frequency, and proximity and clustering of terms. But this is just the beginning. Legal research platforms utilise numerous other factors to influence relevance ranking of documents.


  1. Content type-specific relevance tuning

Firstly, results should be grouped on the basis of content type in order to ensure an ‘apples to apples’ comparison of documents.


  1. Boolean and natural language searching

Both strict Boolean and ‘natural language’ search options have their place in the legal research landscape, and both should be offered to the user.


  1. Phrase recognition and protection

Phrases can either be expressed explicitly by the user, with quotation marks, or recognised by the engine on the basis of their inclusion in a legal phrases dictionary.


  1. Query pattern recognition and ‘target shooting’

Most search engines will have the ability to recognise specific patterns within queries, such as a case name, citation, or legislation title, and treat these as ‘target shooting’ queries.


  1. Activity score boosting

Documents can be given a boosted relevance score on the basis of how many other documents link to them. The number of ‘in-links’ to a document is sometimes referred to as that document’s ‘activity score’.


  1. Document section weighting

The contribution to a document’s relevance score from a hit within its content should be weighted on the basis of the section of the document in which the hit occurs. Armed with an advanced technology platform, lawyers can leverage Big Data to access relevant and comprehensive results when conducting their research.


You can download the full whitepaper at info.lexisnexis.com.au/TechWpDownload Lexis Advance is LexisNexis’ new online legal research platform. For more information call 1800 772 772 or visit www.lexisnexis.com.au/lexisadvance

Recommended by Spike Native Network