This project employs Natural Language Generation technology to present information to patients in an accessible manner. Our prototype system generates short medical dramas from episodes in a patient's medical history. In the script that is generated, a junior and senior nurse discuss the results of tests, causal links between test results and diagnoses, the meaning of medical terms, and so on. The actors are animated agents built and customised for us by Cantoche. Their voices are produced by Loquendo text-to-speech.
Our research focuses on how to make the generated dramas more accessible. In a pilot study with colleagues, the main outcome was that the dialogues our system produced were difficult to follow because the medical information was too densely packed. We came up with four solutions to this problem; each slows down the rate at which information is communicated:
We have implemented these solutions and the generator now produces dialogue scripts that are more socially interactive, so that the interest of the viewer is (hopefully) captured and the medical information more easily assimilated.
The movies, below, show outputs from our medical drama script generation system:
 (23,334KB) |
 (15,172KB) |
| Click on the images to view the movies. If you cannot see them, you may need to Download the TechSmith Screen Capture Codec. | |
Definition Questions
Our script generator generates What is? questions and answers that define medical terms.
Junior: Lymphadenopathy?
Senior: Lymphadenopathy is a swelling of the lymph nodes which the doctor can feel when you are examined.
Senior: The commonest cause is infection, but swelling can be caused by a build up of cancer cells.
Questions about the semantic relations in a patient's medical record
Senior: What was the result of the blood test?
Junior: A low haemoglobin count.
Requests for repetitions
Senior: An excision is the removal of the cancer along
with a healthy border of tissue around it.
Junior: Come again please?
Senior: An excision is the removal of the cancer along
with a healthy border of tissue around it.
Questions that check the addressee's understanding
Senior: Anaemia is a condition in which patients feel
very tired and may become breathless.
Senior: Do you understand?
Questions that check the speaker's own understanding
Junior: Junior: Axilla is the armpit, isn’t it?
Pronunciation questions
Junior: Lympha, lympha...
Senior: Does it say lymphadenopathy?
Junior: Yes.
The application generates emotional responses to medical information. The junior nurse is a student who is fascinated by medical technology and new medical facts. He responds enthusiastically ("amazing!", "cool!") to new facts and technology descriptions. However, when he hears information about patients' suffering, he responds in a neutral way "Right.", "Okay."
Senior: Radiotherapy targets cancer cells.
Junior: Amazing!
Senior: Anaemia is a condition in which patients feel very tired and may become breathless.
Junior: Right.
Either the senior nurse or the junior sums up a medical episode that has just been discussed. From the senior nurse, these are authoratative summaries, whereas from the junior,they are more tentative.
Senior: So, a packed red cell transfusion was administered to treat anaemia.
or
Junior: So, let’s hope that the packed red cell transfusion took care of the anaemia.
Multi-tasking adds material to the dialogue that is engaging, but not demanding, for the viewer. Indeed, experts on writing fictional dialogue scripts recommend that authors introduce sub-dialogues about other subjects into conversations that convey information that is vital to the plot. We introduced a multi-tasking dialogue planner that "mixes in" sub-dialogues about everyday topics with the machine-generated dialogue to break up and slow down the flow of technical medical information. We have hand-crafted a demo showing an example of a such a drama:
Sandra Williams, Richard Power and Paul Piwek [2008] Simulating emotional reactions in medical dramas. To appear in proceedings of the Symposium on Affective Language in Human and Machine, AISB 2008, April 2008.
Sandra Williams, Paul Piwek and Richard Power [2007] Generating monologue and dialogue to present personalised medical information to patients. Proceedings of the 11th European Workshop on Natural Language Generation, pp. 167-170. pdf (401KB)
Paul Piwek, Richard Power and Sandra Williams [2006]. Generating scripts for personalised medical dialogues for patients. Technical Report 2006/06. Computing Department, The Open University. ISSN 1744-1986. pdf (87KB)
This work was funded partly by the EU Semantic Mining Network and partly by the UK Medical Research Council under CLEF Services.
Page updated: 28 February 2008, by Sandra Williams.