Portable smartphone-based fluorescent microscopes are becoming popular owing to their ability to provide major functionalities offered by regular benchtop microscopes at a fraction of the cost. However, smartphone-based microscopes are still limited to a single fluorophore, fixed magnification, the inability to work with a different smartphones, and limited usability to either glass slides or cover slips. To overcome these challenges, here we present a modular smartphone-based microscopic attachment. The modular design allows the user to easily swap between different sets of filters and lenses, thereby enabling utility of multiple fluorophores and magnification levels. Our microscopic smartphone attachment can also be used with different smartphones and was tested with Nokia Lumia 1020, Samsung Galaxy S9+, and an iPhone XS. Further, we showed imaging results of samples on glass slides, cover slips, and microfluidic devices. A 1951 USAF resolution test target was used to quantify the maximum resolution of the microscope which was found to be 3.9 μm. The performance of the smartphone-based microscope was compared with a benchtop microscope and we found an R2 value of 0.99 using polystyrene beads and blood cells isolated from human blood samples collected from Robert Wood Johnson Medical Hospital. Additionally, to count the particles (cells and beads) imaged from the smartphone-based fluorescent microscope, we developed artificial neural networks (ANNs) using multiple training algorithms, and evaluated their performances compared to the control (ImageJ). Finally, we did ANOVA and Tukey's post-hoc analysis and found a p-value of 0.97 which shows that no statistical significant difference exists between the performance of the trained ANN and control (ImageJ).