Autonomous learning algorithms operate in an online fashion in dealing with data stream mining, where minimum computational complexity is a desirable feature. For such applications, parsimonious learning machines (PALMs) are suitable candidates due to their structural simplicity. However, these parsimonious algorithms depend upon predefined thresholds to adjust their structures in terms of adding or deleting rules. Besides, another adjustable parameter of PALM is the fuzziness in membership grades. The best set of such hyper parameters is determined by experts' knowledge or by optimization techniques such as greedy algorithms. To mitigate such experts' dependency or usage of computationally expensive greedy algorithms, in this work, a meta heuristic-based optimization technique, called the multimethod-based optimization technique (MOT), is utilized to develop an advanced PALM. The performance has been compared with some popular optimization techniques, namely, the greedy search, local search, genetic algorithm (GA), and particle swarm optimization (PSO). The proposed parsimonious learning algorithm with MOT outperforms the others in most cases. It validates the multioperator-based optimization technique's advantages over the single operator-based variants in selecting the best feasible hyperparameters for the autonomous learning algorithm by maintaining a compact architecture.