QueueMetrics includes a Quality Assessment (QA) module that lets you:
- Define a set of metrics to be used for call grading
- Have the QA team grade calls while they’re being processed or from historical recordings
- Run complete reports by queue and by agent
In order to use QA monitoring, you should have the following security keys assigned:
- QA_TRACK: this key means that the person can input QA data. If this person has the keys to access historical calls or real-time calls, he will be able to fill-in QA forms. Individual forms can be further restricted by key-protecting them
- QA_REPORT: this keys means that the holder can access QA reports. Individual forms can be further restricted by protecting them with a reporting key as well.
- USR_QAEDIT: this key means that the holder can modify and create QA reporting forms.
The QA module in QueueMetrics was built in order for a specific QA supervisor to track the performance of agents on a given set of metrics. Each metric is expressed as a long description and has an unique engagement code (a short acronym up to 5 letters).
Metrics are user-definable and are clustered together in forms; a form can hold up to 130 metrics divided in up to 10 metric groups.
A single reporter can grade a call only once for each defined form; any attempt to grade a the same form for the same call multiple times will not be accepted.
For security reasons, call grading data cannot be modified once input, and forms with live data associated to them cannot be deleted from the system. In order to have a reduced set of metrics available if you use successive versions of a form over the period, it is possible to close a form, i.e. to avoid further input. Deletions, if any, will be performed at the database level by the system administrator.
Grading data is expressed as integer numbers between 0 and 100; grading all fields is mandatory, except for fields marked as "optional" in the form definition. The QA team can also input free text comments linked to a specific call.
It is possible to edit thresholds for different levels of QA grades, e.g. 0-25: Issue, 26-50: Improvement required, 51-75: Meets expectations, 76-100 Exceeds expectations. These values can be defined on a form-by-form basis, and make it possible to count the number of items that belong to each category and to use a colour code for immediate graphic representation.
Grading data can be input while listening to the live call (Unattended monitoring) or while looking at the historical call details or through a particular formatted URL string.
If an agent has the required grants, he will get the grading icon on the right of the "Calls being processed" table:
The icon appears only when a call being processed is connected to an agent (as the point is rating the agent). By clicking on that icon, a popup will appear that lets you enter QA data. You should be listening to the call using the Unattended Monitoring icon in QueueMetrics or a different passive listening schedule as set up in your call center.
In order to do the grading of historical calls, you proceed as is the case for audio recordings. If QA grading is enabled, the button "Track QA" will appear on the call detail popup, as in the picture below.
By clicking on the button, the call detail popup will close and a QA form will load.
If multiple QA forms are available for this QA person, they will be able to select the correct form by selecting the "Input form" field on the top right.
The top-left box shows the current threshold values for each levels. While you input data into the form, you can see that the number of items that fall into each category and the average and total scores are updated in real-time.
On the top-right box, after the Input form field, a form Status dropdown reports the current form status. Actually this dropdown is read only and their status could be changed modifying the dataset stored in the database with an external application. Following the form status you can see the call details: when it started, on which queue it was processed, the agent processing it, the caller-id and the Asterisk internal call-id.
On the top right there are three buttons:
- Close is used to exit the form without saving changes
- Notes is used to toggle between scores and notes associated to the call
- Print is used to start the print process. The print procedure will show scores and notes on the same page.
Then a general purpose "timer" widget is shown. You can use it, for example, to track the hold time when listening to a call conversation.
At the bottom of the form is the button Save, that is used to exit the form saving changes. This button will appear only when data are in a consistent state that allows saving.
If you load a saved form, it will be shown in read-only mode.
On the bottom part of the form there are the different items to be graded, grouped into a set of categories. If a box contains invalid data (i.e. something that is not a number between 0 and 100 included) it will be displayed in yellow and the form shall not be saved.
Following the form definition, items can be graded by:
- inserting a score value
- selecting the appropriate value from the dropdown menu
- checking the proper Yes/No options.
Items that are not mandatory have an associated N/A checkbox; when checked, it disables the related score value and lets the user save a form without specifying any score for that item. If all fields within a given section are defined as N/As, then the Overall Performance will display the entire section with an N/A Average total value.
Items reported in italic are shortcut items; that is, if a shortcut item totalizes a score that falls into the "Issue" category, the overall form score will be set to 0.
The value set in some items may control the set of items that are enabled for the current form; that’s why the form is evaluated again after each user input.
For a thorough description of how Forms and Items can be set up, please see the chapter Configuring QA forms Section 20.9, “Configuring QA forms” .
When a form is saved, it appears as per the following figure:
It basically shows the same data that was input, but it cannot be changed anymore and the supervisor information is shown. If there are known audio recordings for this call, they are shown under the "Audio recordings" box.
The form now displays a Toggle N/As button which allows to show/hide the Non Applicable specified fields.
Pressing the Notes button, the form will change as per the following figure:
The user can insert one or more notes that will be saved by pressing the "Add" button; all comments already added are listed in chronological order.
![[Note]](../images/icons/note.png){kind=icon} | Note |
|---|---|
Is not possible to submit partial forms. If you compile a partial form, switch to the note view and submit a note, the scores already compiled will be lost. |
Is possible to grade a particular call through an HTTP request to the QueueMetrics server with an URL specifically formatted for this purpose.
When an URL is typed in the browser, QueueMetrics redirects its output to the login page (if required) where the user could log in to continue.
QueueMetrics shows the grading input form in the browser window and the user can grade the call and/or add notes to it.
The URL to be used to trigger the grading procedure should follow the syntax below:
http://qmserver.corp:8080/queuemetrics/qm_qa_jumptogradepage.do? QAE_astclid=1286184814.122 &QAE_queue=queuename &QAE_formName=FormToBeGraded &QAE_CallStartDate=2010-10-04.11:00:00
(of course the URL should appear all on one line).
In the example, we trigger a grading procedure on the host ’qmserver.corp’ on port 8080 The context is queuemetrics (but could change based on local install). Then there are some parameters following:
- QAE_astclid : specifies the Asterisk unique id for the call to be graded
- QAE_queue : specifies the queue name where the call has been taken
- QAE_formName: specifies the name of the form to be graded
- QAE_CallStartDate: specifies the day where the call has been taken. The value should be formatted as YYYY-MM-DD.hh:mm:ss and should represent the time before the call (it’s not important to specify the exact time where the call has been taken but it’s important to specify a time near the period before the call).
Since version 1.7, users holding the key ’QA_REMOVE’ can delete a form.
When a form is deleted, their content is dumped on the Audit Log.
All accesses on deleted forms are highlighted by a special message shown in the form.
After deleting a form, it is again possible to grade a call as if it was never graded before.
In order to run QA reports, you must go to the main page of QM and click on the "Run QA forms" label.
The system will show the following form:
The parameters have the following meanings:
- Form is the name of the form you want to run a report for
- Queue can be one or more queues. You can the run different reports for different queues, or use a catch-all queue
- Agent is an optional Agent filter
- Location is an optional Location filter
- Agent Group is an optional Agent Group filter
- Grader is an optional parameter that filters by the person who compiled the form
- Supervision is an optional Supervisor filter
- Outcome is an optional call outcome code filter
- Start Date and End Date are about the start time of the calls which QA forms that will be included in the report.
By clicking on "Calculate" or "Show Summary" the actual results are shown.
It is also possible to run a report that compares graders to each other - see Section 12.4, “Grader calibration reports”.
The button "Calculate" shows a report like:
The Tracked calls per agent report shows:
- The total number of calls that were tracked for each agent
- The average score for each agent
- The total number of items that fall into "Exceeds expectations", "Meets expectations", "Improvement required" and "Issue" for each agent.
As you can see, the names of each agent are clickable in order to obtain a detail of calls by agent.
All statistics that are computed per-agent are then recomputed per-queue and per-agent-group.
The Analysts tracking calls reports shows how many calls each supervisor graded and what was the average score that this supervisor gave.
Then, for each Section defined in the QA form, you will get the average scores for each item, plus an average of all average scores in order to point out problems.
If an item is shown in red, it means that such item has been assigned a zero-weight value. For further information on configuring items within the form, refer to the paragraph xref:Configuring QA items[].
All columns can be sorted by clicking on the item name and all data can be downloaded in Excel, CSV or XML format.
In order to have a better understanding of what is going on, you can click on an agent’s name and get the details, as below:
This shows the details of all calls stored, the number of items for each call that fall into each grading category, the average rating for each call and the comment.
By clicking on the form icon
on the right, you can access the QA form that was graded for this calls,
so you can access individual scores and listen to
audio recordings that are related to this form.
The "Show Summary" button shows a report like:
This report calculates, for each item and for each section in the form:
- number of calls graded
- average score and cumulated percentage for each item and section that "Exceeds expectations" (the column marked "Best") or "Meet expectations" (Good) or "Improvement required" (Ok) or "Issue" (Req.Imp.).
- cumulated percentage for each item marked as "shortcut"
All values are computed accorded to the currem item weights, in case you use weighted items.
The data can be exported to Excel, CSV and XML formats.