FEATURES

This document collects feature families that were not contained in the optimal combination of feature families in the 12-fold cross-validation evaluation on the standard reference. In our research they were treated on a par with the four feature families that comprise the optimal combination.

1.FEATURE FAMILIES

1.1.Feature family – Structural properties

To identify the section of a findings sentence , we introduce a binary feature that returns if the findings sentence in instance is from the Clinical history section and if it is from the Findings section.

Negations in medical documents are often used to discuss the absence of findings. So despite high similarity between two sentences, dissimilarity can depend on one negation word. To model the presence of negations in either of an instance’s sentences, we introduce the following feature:

where is if the sentence contains a negation word (e.g. “not”, “no”, “negative”), and otherwise.

We discriminate two temporal orientations of a sentence: backward looking and forward looking. In the context of a radiology study that is reported on by a report, a sentence from that report is backward looking if it addresses the time interval before study’s moment in time. Backward looking sentences typically discuss findings on previous exams or clinical history. Forward looking sentences are defined similarly with respect to the time interval that follows the study’s moment in time. Recommendations are typical examples of forward looking sentences.

The temporal orientation of a sentence is related to its tense, but this relation is not one to one. For instance, the sentence “The patient smokes approximately seven packs of cigarettes per week” is backward looking even though it is stated in the present tense.

A sentence can be either forward looking or backward looking, or neither of the two; it cannot be forward looking and backward looking at the same time. Thus we define the problem of determining the temporal orientation of a sentence as a classification problem with three temporal orientation values: (“backward”), (“forward) and (“neither”). For a given function that maps a sentence to its temporal orientation value, we define

Since no obvious syntactic properties stand out to define the function on the basis of a set of rules, we implemented it as a maximum entropy classifier. To this end we manually labeled 1,957 findings and conclusion sentences from the standard reference (i.e. the corpus of 444 reports) with a temporal orientation value. We call the resulting set the ground truth. Each sentence was described by means of four feature families:

  • Words: Each word in the corpus defines another binary feature. If the sentence contains a given word, it has value on the feature corresponding to this word.
  • First word: The first word of each sentence in the corpus defines another binary feature. If the sentence starts with a given word, it has value on the first-word feature corresponding to this word.
  • Date: One feature that returns if the sentence matches a regular expression that covers date patterns, and otherwise.
  • Forward markers: One feature that returns if the sentence has non-empty intersection with a list of forward looking markers (e.g., “suggested”, “recommendation”), and otherwise.

In Table 1 we compare the predictions of the maximum entropy classifier to the ground truth in a ten-fold cross-validation evaluation. We see that the classifier confuses backward and forward orientation eight times out of 488 instances. Accuracy of the classifier is 0.963 ().

Ground truth
Backward / Forward / Neither / Sum
Maximum entropy predictions / Backward / 273 / 4 / 59 / 336
Forward / 4 / 207 / 34 / 245
Neither / 20 / 15 / 1,341 / 1,376
Sum / 297 / 226 / 1,434 / 1,957

Table 1. Comparing the predictions of the maximum entropy classifier against the ground truth in a ten-fold cross validation evaluation

1.2.Feature family – Domain knowledge

The final family of features models particular properties of the breast cancer radiology domain. The first two features in this family measure the degree to which the sentences and report benign findings and if those degrees are aligned. This could prevent the classifier from relating a findings sentence reporting a benign finding to a conclusion sentence thatcontains a non-benign finding.

In a given set of sentences, let be the subset whose sentences have non-empty intersection with a predefined set of “benign” words such as “normal”, “fibroadenoma” and “cyst”. The agreement between the sentences of instance with respect to them reporting a benign finding is defined as

The malignant counterpart of this feature is defined as

where denotes the subset of sentences that have non-empty intersection with a predefined set of “malignant” words such as “malignancy”, “carcinoma” and “suspicious”.

Statements about the left side of the body (e.g. left breast and left axillary region) are unlikely to pertain to statements about the right side. Therefore we introduce a feature that detects if the lateralities of two sentences match. To this end we introduce a function that returns if the sentence contains the word “left”; if it contains “right” and not “left”; if it contains “left” and “right” or if it has non-empty intersection with a pre-defined set of words referring to both sides of the body, such as “bilateral”; and in all other cases. Then,

Dominant imaging modalities in breast oncology are mammography, ultrasound and magnetic resonance imaging (MRI). To account for the modality that reveals the finding reported in a sentence, we introduce the function that returns , and if has non-empty intersection with a list of “mammogram”, “ultrasound” and “MRI” words (in this order), respectively. If has empty intersection with each of these lists, we iteratively compute the value of the precursor of until we hit a sentence that has non-empty intersection. If no such sentence exists, we let . Thus we obtain the final feature:

Unlike the laterality feature, the modality feature looks back to preceding sentences. The reason is that laterality indicators are relatively frequent in sentences that report a finding, whereas modality indicators are only stated at the outset (“The patient was taken to the ultrasound suite”) and no explicit reference to the used modality is made in subsequent sentences.