Dr. Yaji Sripada is the Chief Development Scientist of Arria NLG and one of the company’s founders. He has worked on natural language generation for the past 25 years, and has published 75 peer-reviewed academic papers.
Yaji is particularly interested in integrating NLG to adjacent technologies such as data analytics and information visualization, so we were keen to get his stance on this very topic.
Writing for AI Business, Yaji gives his expert view on the crucial field of big data analytics, explaining:
Why Automating Narratives is Key to Understanding Big Data
A lot of data. A lot of analytics. A lot of dashboards. All aimed at supporting companies in their quest to make sense of business data. Yet the moment of clarity, when true insights are revealed from data, still eludes even sophisticated organizations. Businesses across the globe have spent millions looking for these transformative moments. But why isn’t it working? Why are companies failing to achieve lasting business transformation from data-driven insights?
The short answer is that companies pay less attention to the business users, treating them often as passive consumers of technology rather than active agents responsible for bringing about true data-driven transformation. Technology should serve business users and change the way they think and act; otherwise the whole effort is pointless. Focusing on business users enables businesses to diagnose the issues they face interpreting big data and driving change. The following issues are faced by most businesses:
- Time Commitment: The first issue for business users is that change is anything but instantaneous. Long established patterns in an organization are not likely to change simply because of analytics and dashboards. Recent neuroimaging studies show the brain regions responsible for behavior change in humans are the same ones that respond to pain. Which means that achieving behavior change in business users is a considerable challenge and businesses should commit longer time spans to drive data-driven business transformation. Initially, time should be devoted to winning the business user by bringing about small but sustainable changes in business users’ workflows.
- Innovating the Workflow: New workflows offer clear opportunities to implement data insights. For example, an account manager would welcome data-driven insights when a client account is ready for renewal rather than under regular operational routine. When business users begin to see direct and immediate benefits of data driven insights to their routine workflows, data-driven revolution is a natural outcome, not a forced exercise.
- Cluttered Dashboards: The next issue is that information displayed on a typical dashboard needs further interpretation. While the amount of analysed data continues to grow, so has the size of dashboards: providing a new complex system to present data, but not a solution for presenting it in a digestible format. Thus, from the business user perspective, cluttered dashboards should be replaced by a new generation of dashboards that provide text explanations that communicate information more directly to business users without the need for further interpretation.
- Using Narrative to Translate Data: Finally, humans are a lot more likely to comprehend information if it is presented as a story. In fact, research studies have shown that brain scans of speaker and listener are identical when a well-crafted story is exchanged. This means, if organizations are keen on bringing about a real change, then the focus should be on telling data stories that business users can connect to and that help them make sense of the underlying information. We often see this in practice where dashboards are used by experts as visual aids to tell the data stories orally to their audiences.
How automation can provide a solution
Using expert time to tell a story with dashboards is not scalable. Is it possible for experts to hand-draw all the pretty visuals required in every dashboard? The true reason for the ubiquitous availability of dashboards is the automation of visuals using computer graphics. Besides, data scientists employed by most organizations are skilled at building predictive models from data, but not necessarily skilled at writing stories. Thus, if organizations want stories from all their data to be delivered on demand to each business user, storytelling needs to be automated.
- A New Generation of Dashboards: With the importance of storytelling, textual narratives acting as the expert voice will drive a new generation of dashboards. These storytelling dashboards offer a seamless integration between the familiar visual features of dashboards and the new textual narratives so that business users could interpret the visuals right within the context of the narrative. It would be possible to perform drill-downs from text-to-text (familiar hypertext links), text-to-graphics (new call-outs to on-demand graphics from a textual narrative) and graphics-to-text (new call-outs to on-demand textual narratives from a graphical display of data). The auto-generation of narratives is made possible by a novel technology called natural language generation. NLG can be viewed as a text creation service that dashboard developers use for creating textual objects very similar to creating graphical objects.
- Natural Language Generation: Natural language generation (NLG) is a field of Artificial Intelligence developing computational models for automatically translating data into textual narratives in real-time – stimulating the way one communicates with another via natural language. The science of NLG involves the systematic study of the subtasks of NLG. A good way of understanding these subtasks is to group the subtasks into the following two categories:
- Sense Making Subtasks: These subtasks are responsible for analysis and interpretation of data to create messages that are then organized into a coherent narrative structure. The computational methods employed to achieve these subtasks are knowledge based. This means that the knowledge required to discover and tell stories from data are captured from subject matter experts and embedded into the software.
- Linguistic Subtasks: These subtasks are responsible for mapping messages from the narrative structure to linguistic structures and optimize them for linguistic cohesion. Finally, the linguistic structures are realized (rendered) into the grammatically valid text.
Storytelling dashboards enable business users to comprehend the story behind the data a lot more quickly and effectively. With the recent advances in NLG, dashboard developers can embed textual narratives as objects in the dashboard very similar to graphical objects and reach out to business users with comprehensible information to bring about a true data-driven transformation.