Criteria to Assess a Randomized Controlled Pilot Study: Response to Letter

As some problems have been mentioned in article "Should We Rely on the Findings of Each Published Randomized Controlled Study?" (1), this letter has been provided to clarify them for journal's readerships.

Nowadays, Randomized Clinical Trial studies have been known as gold standard studies on condition that researchers pay attention on the most important factors in design such as:

1. Maximize statistical power in equal group sizes especially in subgroup analyses.

2. Minimize selection bias when randomization procedure will be unpredictable so that researchers cannot guess the next subject's group assignment based on prior treatment assignments. Also it is noticeable that balancing both known and unknown prognostic factors in the assignment of treatments is basic to approve it.

3. Minimize allocation bias when the treatment effect is confounded with the effect of the covariates.

4. Rectify statistical error of RCTs that contains both type I ("false positive") and type II ("false negative") (2-5).

It should be noted there are some other points in these studies that are in the second priorities, which can improve results, but it is conventional like biostatistics procedures. Many consider T test, ANOVA, and Chi Square test were developed over more than 60 years ago and the statistical methodology has been advanced. While these tests may still be appropriate, it is likely that additional statistical analysis will be considered as required analysis by many peer reviewers of journals (6). If we are interested in measuring dependent variables at the beginning and the end of the study at least twice and to compare treatment groups, not just looking at whether there has been any time dependent change in each group of subjects, we would have a multivariate repeated measurement especially when experimental hypotheses are stated in main topic rather than simply main effects or interactions. This statistical analysis is able to determine that there is a change in the variables levels of subjects over time (time effect) within-subjects, treatment effect between-subjects and the effect of time the same for both treatments. Therefore, with regarding to these advantages, authors are keen on using repeated measures in this study. Sometimes investigators detect effects in their experiment that were not expected or decide to explain almost any pattern of means, it would be appropriate to analyze and evaluate that pattern as multiple tests post-hoc. However many journals have recommended that mention of post-hoc comparisons should be avoided in the text (7). Otherwise, it is obvious that researchers have done many statistical analyses such as determining normal distribution of variables by valid methods or establishing the agreement between observers and intra-observer but it is not essential to report all details providing that journals are captivated to express the specific information. It should be noticed this trial study like the others is registered in the clinical trial registry (IRCT), although the journal does not have any tendency to report it. In addition, it should be pointed out that cut-off definition of some variables such as abnormal FBS or liver function test is health based and is associated with progress of metabolic syndrome components neither calculate on population percentiles. Despite authors have paid careful attention to write results, there are some errors. We did not have any missing data and Table below shows correct numbers and percentages. Finally authors appreciate everyone who pays attention to our randomized clinical pilot study and offers the more attention in design of these studies which can lead to achieve the more effective results as we have concerned in this study.