This study investigated reasons why children with attention deficit hyperactivity disorder (ADHD) or arithmetic learning disorder (ALD) have difficulties in solving arithmetic word problems. In particular, the aim of this study was to verify whether these difficulties are due to a working memory deficit and defective inhibition of irrelevant information included in the problem wording. Furthermore, the study was geared to test whether children with ADHD or ALD have a specific disability in recalling and handling numerical or literal information. In an attempt to provide an answer to these questions, three groups of children were tested: children with ADHD, children with ALD, and a group of children achieving at normal levels. They were presented with a battery of arithmetic word problems containing irrelevant information (using either numerical or literal information). Children were asked to recall relevant information within the texts and then solve the problems. Children with ADHD recalled significantly more irrelevant literal information. Both children with ADHD and ALD recalled significantly more irrelevant numerical information. On the other hand, in the phase of problem solving, children with ADHD were more impaired by irrelevant literal information (which overloads memory), whereas children with ALD were more impaired by irrelevant numerical information (which may elicit the execution of wrong arithmetic procedures).