user
@adaoc
Evaluating learner responses
I downloaded the template for generating scenarios and evaluating responses using AI. Is there a template available without scenario generation: just feedback?

I would like the learners to write a reflective text and receive wider feedback than a score: other elements they could have included, who to consult, etc.

Is it possible?
reply
Reply
user
@support
Hello,

Not sure if you got a chance to look at the triggers - you provide the LLM with instructions on what to evaluate. See step 33 in this tutorial. Where we currently ask "Does the customer service message include a greeting?..." etc. - naturally, if you ask for another kind of evaluation, AI will perform another evaluation for you.

Feel free to reach out if additional assistance is needed.
reply
Reply
user
@adaoc
Hello,

Thanks for your reply.

Yes, I looked at the triggers: as you say, they currently do yes/no scoring for a series of given elements, which is not quite what I'm after.

On the one hand, I'd like to know if there is a template available that performs the kind of scoring above without the scenario generation: just the feedback task.

On the other hand, although I didn't specify that in my original question, the kind of feedback I'd like to achieve is not binary scoring: I'd like the AI to also provide suggestions on improvement. The current triggers are not set up for that.

I have written a prompt to instruct the AI on the evaluation task I'd like it to perform given the learner's input but I'm struggling to make sense of which of the three chatbot options would be best for this use case, since learner will not be asking a question.
reply
Reply
user
@stenciluser
Hi. To get a template without scenario generation, all you need to do is remove the one JS trigger that generates the scenario, that's all.

If you want, use the Custom Development portal to send us your .story file and a description of what's needed, and we'll update the JavaScript for you, so you'll have exactly what you're looking for.
reply
Reply