Label-free separation of viable cancer cells using vortical microfluidic flows has been introduced as a feasible cell collection method in oncological studies. Besides the clinical importance, the physics of particle interactions with the vortex that forms in a wall-confined geometry of a microchannel is a relatively new area of fluid dynamics. In our previous work [Haddadi and Di Carlo, J. Fluid. Mech. 811, 436-467 (2017)], we have introduced distinct aspects of inertial flow of dilute suspensions over cavities in a microchannel such as breakdown of the separatrix and formation of stable limit cycle orbits for finite size polystyrene particles. In this work, we extend our experiments to address the engineering-physics of cancer cell entrapment in microfluidic cavities. We begin by studying the effects of the channel width and device height on the morphology of the vortex, which has not been discussed in our previous work. The stable limit cycle orbits of finite size cancer cells are then presented. We demonstrate effects of the separatrix breakdown and the limit cycle formation on the operation of the cancer cell separation platform. By studying the flow of dilute cell suspensions over the cavities, we further develop the notion of the cavity capacity and the relative rate of cell accumulation as optimization criteria which connect the device geometry with the flow. Finally, we discuss the proper placement of multiple cavities inside a microchannel for improved cell entrapment.