The characterisation and comparison of pencil markings on paper is an area of questioned document analysis that has previously not received much attention. Despite this, there would be value in an examiner being able to analyse two pencil markings and coming to a conclusion about whether they were from a similar or different source. Previous studies have analysed raw materials and bulk pencil cores for purposes of characterisation and differentiation, but to date, no studies have successfully analysed pencil markings non-destructively off a paper substrate. In this work, pencils from a number of manufacturers were analysed by inductively coupled plasma mass spectrometry (ICP-MS) and time-of-flight secondary ion mass spectrometry (ToF-SIMS). Analysis of data using multivariate techniques (principal component analysis), showed that ToF-SIMS can successfully be used to analyse such pencil markings to deduce whether two markings can be differentiated, in terms of inorganic elemental composition. It was possible to discriminate between pencil markings from different manufacturers, and it was also indicated that pencils from the same manufacturer, but discrete batches, can be significantly different.