Advanced tracking of agent performance lets you pinpoint those agents whose behavior can and should be improved. In order to do this, a score is created out of multiple items related to:
- their call handling (e.g call duration)
- their call performance (e.g. number of sales)
- QA scores that were given to their calls
As the scoring is in itself quite complex and made up of multiple factors, scoring is based on a rule set that represent a business-specific set of targets that should be met. For each rule, you have two possible levels of non-compliance that is a yellow and a red threhold. Each threshold can, in turn, have a peculiar score associated.
![[Note]](../images/icons/note.png){kind=icon} | Note |
|---|---|
For example, you could say that the expected call duration is 100 seconds; calls that are between 100 and 150 seconds are "yellow" and worth 1 review point, calls that are over 150 seconds are "red" and worth 4 review points. The higher your review score, the more prominently the agent will be displayed. |
When applying a rule set to a set of calls, you get a score expressed in review points for each agent selected that represents the sum of all anomalies as detected by the chosen ruleset.
The system then displays the agents involved in reverse score order, prompting the grader to investigate further by accessing the set of calls and the set of QA records and the relevant audio recordings.
The result of this activity is:
- In-depth knowlegde of agent performance
- Agent life cycle management: the grader can move agents between agent groups, so that you can manage a process where an agent belongs to multiple skill groups during their lifetime
- Continuous improvement of agent performance through agent tasks, e.g. coaching sessions, or completing Commputer Based Training to improve the agent’s skills.
For example, an agent could start her life as member of the group ’New Hires’. When reviewed after a while, she could be moved to ’New Hires Probation’ when she is found lacking in some subject. After a while she could be checked again and moved back to ’New Hires’.
As collateral features, the system also offers facilities to:
- Create rule sets based on the average properties of a set of calls. This makes it easy to have reference points that can then be manually edited.
- Track the lifecycle of agents the agents groups they have been members of and the time period they have been there.
Just like for agents, there is also the problem of comparing graders to each other, in order to have a "fair" view of what is going on and to make sure that grading happens under the company’s guidelines and not each grader’s own preferences. Grader calibration reports fulfill this purpose by comparing graders to each other.
For users holding the key "QA_PERF_TRACK" a new link appears in the QueueMetrics home page, as shown below:
When clicking on it, you are lead to the main search page:
This page lets the grader search for a set of agents to be reviewed. This requires setting three search dimensions:
- A queue (or set of queues) and a time period
- A way to search for a set of agents (a specific one, or a group, or a location, or all agents that have the same supervisor).
- A QA form to be graded
- A rule set (see below) that applies to the above search and defines scoring. You should define your own before you start this activity (see below).
The scoring rule is usually associated to a particular queue and form but the user can override this selection by checking the option "Override queue and form selections" and by specifying other parameters that affect the calculation, like:
- the minimum expected score;
- the minimum calls that should be analyzed,
- the minimum days the agent had to be in the group at the run period specified for the analysis.
These minimums are to avoid considering agents that are undersampled (e.g. if an agent has been scored only once, we can expect this score to be less meaningful compared to an agent whose score is based on 10 elements).
The button "Search" starts the calculation process and a new page will be displayed:
The items shown here are averages on all the calls that were found in the current set. The selected score rule is used to compute the overall Score value, and agents are shown sorted by their score in descending order.
At the bottom of the main table result, a second table shows the agents (if any) that were not included in the report and the thrsehold that was not met.
You can then select each agent in order to access the details. They are reported in a different page, like the one shown below.
The details page is split in two parts. The top part reports the score details for each call the agent answered. The bottom one shows the detailed history associated to that particular agent.
Each line in the top table reports the score calculated by the not-averaged rule, selected in the search page, and other relevant information for each call.
An icon representing a pencil is shown if the call has a QA form associated with it; by clicking it, the associated QA form will be shown in a separate pop-up dialog.
By the bottom of the page, the grader can take remedial actions using the form displayed below.
In order to move the agent to a different group, the grader has to select the new group through the dropdown; he can specify a reason in the lower text box then press the OK button on the right side of the dropdown group.
If the user checks the "Remind me" checkbox before pressing the OK button, QueueMetrics will send a reminder task to the grader himself that will be displayed after a specified number of days. (This can be used as a reminder and is optional).
A new row with the operation details will be inserted in the agent’s history table after completion.
In order to send a CBT (Compute Based Training) to the selected agent, the grader has to insert a text title in the CBT text box and a valid URL in the Reason text box, then has to press the OK button on the right side of the CBT text box.
A new ’Teaching’ task will be sent to the agent with the title and the inserted URL.
A new row with the operation details will be placed in the history table after completion.
As explained above, in order to track performance you first have to express a set of business targets that express what is expected from your agents and how much deviations from each rule are comparatively worth, expressed in review points. This is called a ruleset.
This can be done through the proper configuration page by users holding the key "QA_PERF_RULES"; they will see a new link from the home page:
Selecting the link, a new page is shown listing rulesets already defined. In order to define a new ruleset, you press the "Create New" button.
The "Create New" button opens a new page where an empty rule is shown, like in the picture below.
| Note |
|---|---|
Targets will not be displayed until you first save the ruleset. |
The creator should assign to the new ruleset a name, a short description, and optionally a security key.
A rule is usually linked to a specific queue (or set of queues) and form. This is because we expect to have homogeneous statistic distributions in the same queue and form items. This might not be true outside a specific form and/or queue. The user should select a specific queue and form before pressing "Save".
When editing a ruleset, you see it is actually built out of a number of targets. It is important to understand that there are basically two different kinds of targets:
- "Aggregated" targets - identified by AVG - that are computed once per agent, and
- "Atomic" targets that are computed for each call handled by the agent
When computing the review score for an agent, first each call is checked against atomic targets and a first score is computed, then averages for the dataset are taken, and they are computed against aggregated targets and an aggregate score is computed; the final score is the sum of both scores.
![[Tip]](../images/icons/tip.png){kind=icon} | Tip |
|---|---|
You can use either type of target, or both as once, as you best see fit. Try and run some tests to make yourself familiar with the ruleset. |
![[Warning]](../images/icons/warning.png){kind=icon} | Warning |
|---|---|
It is important to note that some targets are not available as atomic targets. Examples are the QCPH, Sales, Number of calls, etc. that are obviously associated to a set of calls and make no sense in relation to a single call. |
For each possible target within the rule set, you can:
- Enable or Disable a specific target rule
- Insert an algebraic expression defining the rule for the "yellow" theshold
- Specify a score that the engine will assign to the target when matching the "yellow" expression
- Insert an algebraic expression defining the rule for the "red" level
- Specify a score that the engine will assign to the target when matching the "red" expression
The algebraic expressions that can be used to define a threshold are:
- simple mathematical expressions formed by an operator (in the set of "<", ">", "⇐", ">=") and a value.
- X << Y: defines the internal interval between the values X and Y (excluded)
- X ⇐⇐ Y: defines the internal interval between the values X and Y (included)
- X >> Y: defines the external interval outside the values X and Y (excluded)
- X >⇒= Y: defines the external interval outside the values X and Y (included)
Valid examples are:
- "< 10" is triggered by a number lower than 10
- ">= 40" is triggered by a number greater or equalling 40
- "40 << 80" is triggered by a number between 40 and 80
For not-averaged rules the user can access a wizard that simplifies definition of interval-based rules.
A rule set can be inferred from the measured properties of a given data set; this basically lets you express differences in terms of a percentage of outliers expressed on the total number of calls.
In order to access the wizard, you click on the "pencil" icon:
In this modal dialog you define a start and end time period and the "yellow" and "red" percentage of calls the user wants to include in the resulting rule, the type of interval (internal or external) and whether the interval extreme values should be included or not in the resulting rule.
| Tip |
|---|---|
Imagine you want to consider "yellow" the 10% of calls that are too long or too short and red the 5% of calls that are way too long or way too short in relation to the average length. You would set the "yellow" slider to "90% external" (meaning you want the external tails) and the "red" slider to "98% external". |
The "Go compute" button runs an internal analysis that reports, in the lower right table present in the dialog, the minimum and maximum values representing the interval fulfilling the inserted parameters and the number of calls analyzed. You can repeat the calculation until satisfied, then press "Save" to insert the rule in the rule-set or press "Cancel" to forget it.
This is a separate, one-page report that is only accessed by supervisors earmarked by the key "QA_CALREP" only (in addition to "QA_REPORT").
To access the page, you go to ’Quality Assessment’ → ’Run QA Reports’ and fill-in the form by the bottom of the page :
You will also use the form by the top of the page you usually use for QA reports.
On the input page you select:
- A date range (dual selector plus predefined periods)
- One Form (if we had an option for All,wed get way too many)
- A queue or composite queue
- A call outcome, or none for all calls
- It is possible to select an agent group as an additional filter.
- It is possible to add a minimum threshold of graded calls per grader to be included
The analysis happens at three levels:
- The whole form
- The section level
- The question level
For each form/section/question, a table is computed for the general and for each agent that has graded at least X items:
For each form/section/question, an average is computed and compared to the one of all graders who graded at least X calls in the specific area. This way it is easy to spot trends and anomalies on grading behavior.