Background: No existing instrument addresses the minimum number of items that guarantee methodological quality in studies based on observational methodology. Consequently, professionals who are not experts in observational methodology do not have a basic framework to guide their practice in this type of study. This study developed a checklist to measure the minimum number of items for methodological quality that studies based on observational methodology should consider and provided evidence of their validity based on test content and intercoder reliability.
Method: Fifty-four judges with at least 1 year of experience in observational methodology and research based on this methodology evaluated the items of the developed checklist in terms of relevance, usefulness, and feasibility. Items were selected if they obtained at least .5 in the Osterlind indexes of the three aspects evaluated. Two coders applied the selected items to a random selection of articles that used observational methodology to investigate soccer, and intercoder reliability was examined using Cohen's kappa (k) coefficients.
Results: The final checklist included 16 items grouped into 11 criteria/dimensions, with adequate reliability coefficients.
Conclusions: This study developed a useful instrument for non-expert professionals to enhance the methodological quality of studies based on observational methodology.