Exploring insight lightly: LiteInsight

[arranzeyuwang]

LiteInsight is the insight-driven visual exploration lib of AVA.

It employs a pruning-based insight exploration model, which classifies data by insight type through subspace enumeration, pruning and pattern matching. Thus calculates its insight score, and outputs insight results, meta-data and optional output structure information. The insight score consists of numerical impact and task-related impact.

The pipeline of LiteInsight can be shown like this:

The output Metadata is consists of core and additional attributes as follows:

1. Core information
   Insight tuples: <Subspaces, Breakdown, Measures, InsightType, Score>
   PatternInfo: model pattern information
   FilteredData: Chart data
   ChartType: Type of insights
   NLG: Output of natural language generator
2. Additional information
   Schema (in G2Plot)
   Annotation (in G2Plot)
   *Non-G2Plot charts should be constructed according to the core information and the characteristics of your own charting library

A exemplar in JavaScript can be expressed as follows.

type Subspace = {
  dimension: string;
  value: string;
}[];

type InsightType = 'outlier' | 'trend' | 'extreme' | '...';
type InsightData = any;
type NLGInfo = any;
type ChartType = 'line_chart' | '...';
type PatternInfo = OutlierInfo | TrendInfo | any;

interface InsightInfo {
  subspaces: Subspace[]; 
  breakdowns: string[];
  measures: string[];  
  insightType: InsightType;
  score: number;
  data: InsightData;
  patternInfo: PatternInfo;
  chartTypes: ChartType;
  description： NLGInfo;
  chartSchema?: Schema;
}

interface OutlierInfo {
  dimensionValue: string;
  measureValue: string | number;
  significance: number;
}
...

For graph data, LiteInsight will provide Graph-based subspace enumeration capability in the future. In which subspaces can exist in the form of both subgraphs and sub-tables. Insight of graph data can be expressed by annotations and layouts.

Back to the Big Picture.

[wanghuanliang]

What does numerical impact, task-related impact, and significance mean and how to calculate them?

[Esorakouki]

Significance and (numerical) impact are concepts defineded in chapter 3.2 in 《Top-K》

1. The significance of an insight type is defined based on p-value by using different kinds of null hypotheses;
2. (numerical) impact reflects the importance of the subject of an insight (a data scope) against the entire dataset. Users can specify meaningful impact-measures aligned with their needs;
3. Task-related impact (TODO) is a factor means the user's personal interest in data scope and insight types.