Simple ancillary metadata, such as those encompassed by the 15 elements of the Dublin Core, may be sufficient and entirely appropriate for basic coarse-granularity cross-domain resource discovery. However, they are insufficient and inappropriate for content description of complex data types such as videos, which require more detailed relational models. We propose a metadata classification schema for the characterization of items and events in videos that permits subsequent query by content. Following MPEG-7 nomenclature, metadata intrinsic to the information content of the video are defined as either structural or semantic, where structural metadata are numerical feature primitives produced by analysing the colour, shape, texture, structure and motion within the video frames, whereas semantic metadata describe the locations and timings of individual items and particular actions or events in the video, and are thus of higher information value. In this paper, the semantic metadata required to describe the visual information content of videos are defined and classified into four distinct classes: Media Entities; Content Items; Events; and Supplementary Items, and three types of property tables are defined: Identity Tables; Spatio-Temporal Position Tables; and Event Tables, in which these metadata may be stored in a relational database.