Project PLeNTaS

Natural Language Processing for the Semiautomatic Grading of Tasks

The current state of the art of text-oriented analysis technologies allows the assessment of non-trivial issues such as the overall cohesion of the writing, the main concepts mentioned in a text or the proactivity of a participant in a forum. This analysis capability, applied to the educational environment, allows for automated text evaluation which, if properly planned and orchestrated, would offer a great support to the task of marking activities by teachers. The expected result is a more agile marking with a more formative feedback for the student.

With this idea in mind, the general objective of this project is the design and development of a software system to support the marking of short answer and online discussion activities supported by forums, analysed in a given context by means of Natural Language Processing and Social Network Analysis techniques. The system shall assign, to a delivered activity, a percentage of compliance in each of the metrics established in the grading criteria. This percentage shall be accompanied by an understandable explanation of the reasons why this value has been assigned.

The system will support the teacher during the whole cycle of the task: design, implementation and evaluation. During the design the teacher will develop a rubric with those aspects he/she wants to take into account in the assessment and the system will provide the metrics and technology to assess the specific aspects. During the execution phase of the task, in the case of the forums, the teacher will be able to see how each student’s rubric is progressing and will also be able to see the evolution of the group. At the end, the system will provide a rating for each aspect of the rubric, as well as a textual explanation of the rating that the teacher can review before it is given to the student. In a typical case, the textual explanation will be selected from a catalogue of pre-established and properly parameterised texts.

The expected outcomes are:

  • A pedagogical model that will allow a solid definition of the typology of the activities, the applicable evaluation metrics and the implications of the system in the life cycle of the activity.
  • A platform that will allow, subject to informed consent, the capture of data in an online teaching scenario and the deployment of the analysis system within the educational scenario itself.
  • A data analysis platform in which the different techniques of Natural Language Processing, Text Mining and Social Network Analysis will be developed and validated.
  • High impact scientific publications.

Partners: Universidad Internacional de La Rioja (UNIR), Telefónica investigación y desarrollo

UNIR teams: School of Engineering and Technology (ESIT), UNIR iTED

 

Project duration: 2021-2023

Funding type: National public