Generating annotated data is important for training an NLP system to automatically identify, encode, and extract clinical information from textual data. In an annotation study life cycle, there are several steps to creating a reference standard of annotated data for developing an NLP system: 1) creating an annotation schema and guidelines for instructing annotators how to annotate the clinical text data, 2) recruiting and training annotators to create a reference standard, and 3) managing the distribution of annotation files including documents and schema files.

  • Annotation Schemas (instruct annotators)
    • ConText - original
    • ConText - extended (Temporality) Long and Short
    • SOAP
    • ShARe/CLEF Task 1
    • ShARe/CLEF Task 2
    • Uncertainty
  • Annotation Registry (recruit annotators for annotating a corpus)
  • Annotation Admin (manage distributed annotation)