Manage KPIs

Apromore allows us to create and manage KPIs on an event log, which can be used to track SLA breaches in a process. To create a KPI in Apromore, go to Portal. Select the event log for which we’d like to create the KPI. Right-click and select KPIs.

The KPIs attached to the selected event log appear. If no KPI has been added to this log, the KPI window appears empty.

To add a new KPI, click Add.

The KPI center window appears, containing existing KPIs associated with the event log.

Click Create new KPI.

The Create new KPI window is displayed.

For a KPI to be successfully created, we need to define the KPI name, KPI population, KPI condition, and KPI target. We can optionally add a description to the KPI.

Enter a KPI name. This name will be used for future reference in the KPI. We can call it Case duration KPI.

Specify the KPI population

The KPI population defines the subset of logs to which the KPI condition will apply. By default, the KPI population is set to Entire log. This means that the KPI condition will apply to the entire log.

We can, however, apply the KPI condition on a subset of the log. For example, if we wish to track only high-cost cases, we can set the KPI to apply to a subset of the log where the case cost is more than 50 EUR. To do that, click on the dropdown next to KPI population. Select Filter.

Click Edit to add the filter.

The Filter log window appears. To create the filter, go to the case Performance tab. In the Performance measure dropdown, select Case cost.

We can use the slider to select cases whose cost is greater than 50 EUR. Click OK.

The filter is applied as the KPI population.

We can edit the filter by clicking Edit or completely delete the filter by clicking Clear.

Specify the KPI condition

The KPI condition is the SLA violation we wish to track. To specify the KPI condition, click the dropdown next to KPI condition.

If our KPI is that cases should be less than 90 days, we specify it using the dropdown. The KPI condition could be Case duration/cost is less than or equal to, Case duration/cost is greater than or equal to, or Filter.

In our case, we choose Case duration is less than or equal to. Enter 90 as the value, and click Days from the dropdown.

We can also create a more customized KPI condition using filters. To create a KPI condition with filters, click Filter from the dropdown.

The KPI condition now displays existing filters that can be edited or cleared. If no filter has been created, the Filter type and Description appear empty. If, for example, our KPI is that the average processing time should be less than 1 hour. We create this filter as the KPI condition. To create a filter, click Edit. The Filter log window pops up.

To create the filter, go to the case Performance tab. From the performance measure dropdown, select Average processing time. Use the slider to select the average processing time range. Click OK.

If we wish to create another filter, click the “+” button in the top left corner and specify the new one.

Once we have created the filter(s) that satisfies the KPI, click OK.

The filter(s) will be applied as the KPI condition.

We can edit the filter by clicking Edit or completely delete the filter by clicking Clear.

Specify the KPI target

The KPI target defines the permissible threshold for which the KPI will be triggered. We can specify the KPI target using the basic or advanced settings.

Define a basic KPI target

By default, the KPI target is set to Basic. In Basic, we can only set a target (also called a threshold) to trigger the KPI. When the threshold is exceeded, the KPI is triggered with the colour red, indicating an SLA violation exists. When the threshold is not exceeded, the KPI is triggered with the colour green, indicating no SLA violations.

For example, we may want the KPI to appear green when the percentage of cases that fulfill the KPI condition is equal to 100. Otherwise, appears red.

To do this, ensure the switch is at Basic and the percentage switch is turned off. When the percentage switch is turned off, values are considered numbers, not percentages.

In the dropdown, we may select less than (<), less than or equal to (<=), equal to (=), greater than (>), or greater than or equal to (>=). In our case, we select less than (<).

In the text box, enter the value. In our example, 100% is the value of the population. We enter 184.

The KPI appears red because the number of cases that satisfy the KPI is 159. The set threshold (184) is not met.

If we change the threshold to any value greater than 150, the KPI will be triggered with green. This is because the KPI result (159) is not less than the minimum threshold we have set (150).

It may also be convenient to set the threshold using percentage and not value. To do this, toggle on the percentage switch . Assuming the minimum threshold is 80%, we set the KPI target to > 80%.

Here, we see the KPI is green because 86.41% of the cases fulfill the KPI condition, which is greater than our set target (80%). If, for instance, we increase our target to 90% or greater, the KPI would be violated and turn red.

Define an advanced KPI target

We can customize the KPI target more by switching to the Advanced mode. In the Advance mode, we can set Lower & upper bounds, Mean + tolerance, or a Custom KPI target.

1. Using Lower and Upper bounds

The lower and upper bounds allow us to define a range of thresholds rather than a single value. For instance, we may say the KPI is fulfilled only if the percentage of cases that fulfill the KPI condition is between 80% and 100% (a range. Otherwise, it is violated. To do so, switch to the Advanced mode and switch on percentages. Go to the Lower & upper bounds tab and enter the upper and lower bound percentages.

The KPI is green since the cases that fulfill the KPI condition (86.41%) are within the range (80 to 100%).

2. Using mean + tolerance

We may also define the KPI target using a mean value and some tolerance. For instance, we may set the threshold to 90% with a tolerance of 5%. This means the range will be 90% - (5% of 90) and 90% + (5% of 90). That is a range of 85.5% to 94.5%.

To implement this, go to the Mean + tolerance tab. Enter 90 as mean and 5 as tolerance.

We see the KPI is green because 86.41% is greater than the lower bound of the tolerance (85.5%) and less than the upper bound of the tolerance (94.5%).

If we change the tolerance to 2%, the range becomes 88.2% to 91.8%.

Since 86.41% does not fall in this range, the KPI is violated with red.

3. Building a Custom KPI Threshold

We can build a custom threshold that allows for more complexity to the upper and lower bounds and the associated color code. For instance, we may decide that the red color appears when the percentage is less than 80%. When it is between 80 to 90%, it is amber; when it is greater than 90%, it is green.

To do this, go to the Custom tab and enter the ranges.

We can add more range and colors if we desire by clicking the “+” sign.

Note

We may delete a range and its associated color by clicking the tick box to the left and the trash bin button .

Add a description if necessary, and click OK to save the KPI.

The KPI is saved and appears in the KPI center. This KPI can be attached to the log by ticking the KPI and clicking Add.

Download and upload KPIs

KPIs in the KPI center can be downloaded as a file. To download a KPI, open the KPI center. From Portal, right-click the log with the KPI and click KPIs. We see the KPIs attached to the log. Click Add to open the KPI center window.

Click the KPI we wish to download and click Download.

The KPI is downloaded as a zipped JSON file on our local machine.

Conversely, we can upload a KPI as a JSON file. To upload a KPI, click Upload in the KPI center.

Select the KPI as a JSON file we wish to upload. The file will be loaded to our KPI center.

Use KPIs in Dashboards

Apromore allows us to use KPIs in dashboards. To demonstrate the use of KPIs in dashboards, three other KPIs have been created and added to the KPI center. The KPIs are named Low threshold KPI, Mid threshold KPI, and High threshold KPI.

To use a KPI in a dashboard, create a new dashboard.

Click the Add KPI icon . It adds one of the KPIs to the dashboard. Click the KPI icon twice to add two more KPIs to the dashboard.

Click on a KPI to edit it. In the Presentation view, click the KPI dropdown to select a KPI we wish to display.

If we click Low threshold KPI, the KPI appears in the dashboard with its color code. We can repeat this for the other KPIs.

The KPIs can also be displayed as a gauge. Select Gauge as the display type.

Note

For a dynamic event log, i.e., when there is a defined ETL pipeline for the event log, the KPIs in the dashboard are automatically updated based on the current event log.

We can customize the visualization in the dashboard further using the settings in the presentation tab.

Use KPIs in Process discoverer

Apromore allows us to use KPIs in Process Discoverer as well. When a KPI is added to Process Discoverer, the KPI overview is displayed. To add one or more KPI to Process Discoverer on the Apromore Portal, double-click an event log to open it. Click the Manage KPIs icon .

This displays a list of added KPIs. If we have created KPIs but have not been added, click Add to add KPIs to the event log.

The KPI center window appears, showing a list of already created KPIs. Mark one or more KPIs to add and click Add.

Now, when we click the KPI icon in Process discoverer, each KPI, alongside its color code, is displayed.

We may add or delete KPIs by clicking Add or Delete, respectively.

Perform root cause analysis

As a process manager or analyst, when a KPI target is not met, we need to understand which cases are more likely to violate the KPI. This information can help us identify the causes of KPI violations and corrective actions.

To help in such investigations, the Root Cause Analyzer of the KPI Center displays a list of conditions under which the risk of KPI violation is considerably higher when this condition holds compared to when the condition does not hold.

For example, given a KPI that states that at least 95% of cases should have a duration below 12 days, the Root Cause Analyzer generates a list of statements such as the following:

The KPI violation rate when “Invoice Amount” >= 50K is 7% versus 3% otherwise.
The KPI violation rate when “Country” is “Canada” is 2% versus 6% otherwise.

The first of these statements indicates a negative deviance: the subpopulation of cases where the Invoice Amount is above 50K needs further attention. The second statement indicates positive deviance: The cases in Canada are handled more efficiently than others, suggesting that there is something to learn from how cases are handled in Canada.

We can sort the findings based on several metrics that are common in the field of risk analysis:

Violation rate: The percentage of cases in the KPI’s population that fulfill the condition and violate the KPI. In the first of the above examples, the conditional KPI violation rate for the condition “Country” is “Canada” is 2%.
Inverse violation rate: The percentage of cases in the KPI’s population that do NOT fulfill the condition and violate the KPI. In the first of the above examples, the inverse KPI violation rate for the condition “Country” is “Canada” is 6%.
Risk increase: The difference between the KPI violation rate and the inverse KPI violation rate. A risk increase of 5% tells us that if a case fulfills the condition, the probability of this case violating the KPI is 5 percentage points higher than if it does not.
Risk ratio: The ratio of the KPI violation rate to the inverse KPI violation rate. A risk ratio of 2 tells us that cases that fulfill the condition are twice more likely to violate the KPI than cases that do not.
Sub-population: The number of cases in the KPI population that fulfill the condition.
Sub-population percentage: The percentage of the subpopulation size to the KPI’s population.

A finding is negative if the risk increase is positive. In this case, the finding tells us that a condition under which the KPI violation rate increases. Conversely, a finding is positive if the risk increase is negative (the finding suggests a condition under which the KPI violation rate decreases).

The Root Cause Analyzer retrieves the findings that are most “informative”. The informativeness of a finding is measured using decision tree mining techniques.

To investigate the possible root causes of KPI violations with the Root Cause Analyze, open the KPI Center on a log (in the Portal or Process Discoverer). Add a KPI to be analyzed.

In our example, we have defined a KPI, “Case fulfillment performance”, that flags cases with a case duration greater than 2 weeks.

Click Investigate next to the KPI to analyze its possible causes.

This opens the KPI Root Cause Analyzer window for the selected KPI.

The KPI Root Cause Analyzer window displays a list of “findings”. Each finding appears as a card and consists of a textual description. The metrics of this finding are risk ratio, risk increase, subpopulation size, and subpopulation percentage. Note that the KPI violation rate and the inverse KPI violation rate are included in the description. This helps us understand specific contributing factors to the KPI violations.

We can sort the list of findings in ascending or descending order according to:

Violation rate
Attribute name
Risk ratio
Risk increase
Sub-population

To change how the findings are sorted, click the Sort by icon.

Select the metric by which we wish to sort.

To sort in ascending or descending order, click the Sort by icon . Select Ascending or Descending.

The findings are now listed by the selected metric in ascending or descending order.