What is Modjo AI Call Scoring ?
With AI Call Scoring, verify at scale if your qualification methodology is being applied by your teams.
Thanks to AI, ensure that your sales teams ask the right questions during each interaction with their prospects. For example, during an outbound call, make sure that your client's needs are properly addressed.
In this way, track the adoption of your qualification methodology by your teams and verify its effectiveness in converting opportunities.
Enable your managers to focus their coaching efforts in the right place and verify the effectiveness of your methodology on transforming opportunities.
How does it work ?
1. How to set-up AI call scoring
A. Getting started
You must be a Modjo Administrator/Manager
You must have enabled the Modjo AI on your organisation
This feature is in BETA version, we will keep improving the accuracy and UX of the features. We rely on your feedbacks, don't hesitate to share them with us !
B. Create a template
Go to Settings / Templates / AI call scoring,
Click on Create a template, or use a default template
Give a Title to your template. For example: "Inbound calls", "Discovery"...
We recommend using AI Call Scoring for the qualification use-cases.
💡 Pro tips : Since AI Call Scoring can be sensitive, we recommend you to create a template named "Test - BANT " until you're satisfied with the questions you've built.
C. Choose which calls to apply the AI call scoring
When editing the template, select the relevant filters (Teams, tags, call duration, call provider) to select on which calls the template should be applied.
EXAMPLE
You create the following template:
Title :
Meeting introductionTeams :
Sales,Account ManagementTags :
Discovery,Onboarding
In order for the template to be applied to a given call, Modjo will check:
Call attendees has 1 user member from the "Sales"
OR"Account Management" teamsAND call has tags "Discovery" OR "Onboarding"
Once the templates are created and published, the AI Call Scoring feature will apply to future calls. Past calls will not be evaluated retroactively.
2. Building questions & measuring accuracy
A. How to write questions ?
You can add up to 10 questions per templates.
Basic rules:
The question should always be evaluating what's been said objectively during the conversation
It cannot track something that is subjective like humour, empathy, time to reply, talk ratio...
It cannot assess how many times something has been done (e.g Did the sales answer to 3 main objections ? )
The good answer to your question should always be YES.
Ai call scoring, provides a completion score on the call, yes will be counted, not mentioned will be negatively impacting the score.
For example It cannot track "How many objections were answered during the call ?"
To evaluate only one side of the conversation, you should you use exclusively these terms:
Sales for looking into your team's speech ( Sales rep, BDR, AM, AE...)
Prospect for the external contacts speech ( prospect, client, candidate...)
Example : Did the sales ask...? Did the prospect mention...? Did the sales ask the prospect what are his pains ?
Languages : You can write down your questions in the same language of your team's calls.
If your team makes calls in multiple languages you can create all template's questions in English, it will work fine.
Single-question : You should ask only one thing by question.
❌ Example : "Did the sales ask the prospect about the budget ? And the ROI expected ?"
In what case would you like the AI to answer yes ? When there is at least one of the 2 options matched ? both ?You'd rather create two separated questions, or be more precise in your phrasing.
✅ Example : "Did the sales ask the prospect about the budget ? For example by asking what budget he can allocate on the project, or what is the ROI expected."
B. Creating prompts: Steps and best practices
What do we want to assess?
Start by asking yourselves what is your sales methodology, and questions you want to assess at scale.
For example, you expect your team to cover the different elements during the discovery call :
Introduce themselves
Mention the call recording
Following the BANT Methodology, you want them to discover the prospect's Budget, Authority, Need, and Timeline.
Mention systematically a premium offer to upsell.
Discuss next steps
Who mentioned it ?
Is it something you want your sales to mention in 100% of the conversation ? or are you satisfied if the prospect mentions the topic by himself, without the sales asking the question ?
Example 1 :
Did the sales mention the call is recorded ? --> needs to be in 100% conversation mentioned by the sales
Example 2 :
Here is a cold call conversation extract :
Sales : "Have you heard about Modjo ?"
Prospect : "I know your product, but we have other priorities this quarter so we won't be interested to discuss before the end of 2024".
If AI call scoring question is : "Did the sales ask the prospect about the timeline of the project?"
👉 AI will answer NO, because sales didn't ask the question. However your sales did good in the conversation.
If AI call scoring question is : "Did the prospect mentions the timeline?"
👉 AI will answer YES
What is your context ?
Would someone that is completely stranger to your company and processes, be able to answer rightfully this question on his own when reviewing a call ?
Example 1 : Did the sales introduce himself ?
What does "introduce" mean for you ? stating his name ? Or more precise elements like also your company name ?
Different version possibles
Did the sales introduce himself ?
Did the sales introduce himself by mentioning his name and Modjo ?
Did the sales introduce himself by mentioning his firstname and lastname ?
...
Example 2 : Did the sales ask about the prospect's budget ?
Is a budget a common concept in your customers interactions ? Is there other way to find that information in the conversation?
Did the sales ask about the budget or the expected ROI during the conversation ?
Did the sales ask the prospect about his income and savings ? (bank B2C)
Example 3 : Did the sales mention the next steps ?
Next steps can be very different things according to your organisation. What do you put behind these words : any action plan ? a meeting booked ? an email sent?
Did the sales mention the next steps?
Did the sales scheduled a second meeting with the prospect?
Did the sales proposed a date for the next interview?
Tips for optimizing your prompts:
Be clear and precise in defining the terms assessed.
Limit questions to a single objective to avoid ambiguity.
Adapt questions to your specific methodology and context to ensure relevance.
C. Validate & improve questions accuracy
Providing feedback on the accuracy of the scoring is a powerful way to enhance the effectiveness of the AI Call Scoring feature.
Why label the questions ?
It will be essential to listen to calls or read transcripts to check the questions and assess AI responses.
Annotated responses establish an accuracy score, a crucial human validation confirming proper question configuration.
How to Label Questions ?
Use
👍 Agreewhen:The AI correctly validated the question and it was well-mentioned in the call, with a sensible justification.
The AI correctly identified a lack of mention in the call.
Use
👎 Disagreewhen:The AI correctly validated, but the justification is un-related to the question.
The AI incorrectly validated, and the element was not mentioned in the conversation.
The AI incorrectly claimed no mention, while it was mentioned in the call.
How to review Accuracy ?
Modjo will compute an "accuracy score" per question, based on total reactions:
Number of 👍 / Number 👎
To view the accuracy:
Open your Template
For each question you will be able to see an Accuracy score.
We consider that an accuracy of 70-80% is a good start to be able to deploy the template with interpretable results.
What should I do when the accuracy is low ?
Open the question, go to the Accuracy & history tab, and look at how many reactions were made on this question.
> Below 20-25, the score is not really interpretable.
After reviewing some calls you should have a better idea of why the question is not performing well.
You should ask yourselves the following questions :
Is the question subjective ? For example asking AI to count occurences, asking AI to qualify is user answered properly to the objections ?
Concept is too vague for the Ai and it doesn't match how your sales discuss it in conversations. Add examples.
Pro tip : You should always ask yourselves, if someone that doesn't know anything about our sales pitch or our process, could answer this question.
3. Analyse scores & coach your team
💡 AI call scores Analytics are only visible by Managers & Admins on Modjo
Once you’ve reached a good accuracy score per question, you can finally get a macro view of your team's performance and deep dive into specific questions to assess the usage of your methodology.
You will find an overall score at the top, indicating the extent to which your teams are applying your methodology overall, as well as the number of evaluated calls related to this average.
Scrolling down, you'll encounter a score per template. This feature enables you to understand which parts of your organisation apply the methodology less consistently.
To see the details for each question, click on the average bar of the template, OR select the template in the filters on the left
The user breakdown will adapt according to your left filters selection.
For example, in the previous image, we filtered by:
The BDR team
AND the 'Inbound 1st call' template
AND the 'Authority' question
You can see here that the authority question is addressed in only 22% of the calls by the BDR team, and you can view the details by individual team member on the right.
Thanks to AI Call Scoring, you can quickly identify the elements of your qualification methodology that are the least mastered and know which topics you need to coach your teams on.