Early detection of dementia as well as improvement in diagnosis coverage has been increasingly important. Previous studies involved extracting speech features during neuropsychological assessments by humans, such as medical pro- fessionals, and succeeded in detecting patients with dementia and mild cognitive impairment (MCI). Enabling such assessment in an automated fashion by using computer devices would extend the range of application. In this study, we developed a tablet-based application for neuropsychological assessments and collected speech data from 44 Japanese native speakers including healthy controls (HCs) and those with MCI and dementia. We first extracted acoustic and phonetic features and showed that several features exhibited significant difference between HC vs. MCI and HC vs. dementia. We then constructed classification models by using these features and demonstrated that these models could differentiate MCI and dementia from HC with up to 82.4 and 92.6% accuracy, respectively.