As I said earlier, including people on benzos (known memory impairing effects) would create bias also.
I believe it would be confouding in this case, not bias. Might seem like nitpicking, but it is really an important distinction in this case. It is generally easy to correct for confounding, once you have identified the confounding parameters. Bias is rather hard to correct for, especially selection bias. A few options to correct for the (possible) confounding created by inclusion of benzo users:
- Display all scores for groups with and without benzo users, to see if including benzo users alters the significance.
- Calculate the relative number of benzo users in each group; more benzo users in the 'heavy MDMA users' group could mean something.
- Use of paired statistical test to reduce the influence of confounders.
- Couple benzo users in the MDMA users group to age/sex/education/etc matched controls in the control group (would be tricky with the low number of participants in the current study though).
Beside the possibility of correcting for confounding, there is also another point to my critique on the study: I do not have to present the perfect alternative. If you publish a study that is guaranteed to be controversial, you should make an effort in reducing or at least identifying your own shortcomings. Studies like this are never perfect, but when critically reviewing a study it is not necessarily the objective to provide a better alternative. The main objective is to identify possible mistakes or shortcomings, so people can decide for themselves if the study and its conclusions are credible or not. The fact that I might fail to provide a better way to do it does not mean this shortcoming has no effect on the credibility of the study.
You are correct in saying that people with severe anxiety, depression, or memory loss are more likely to be on medication than healthy people, but do not expand on why the exclusion of people with possible preexisting medical conditions undermines the strength of the conclusion (with the exception of saying that a selection bias is created, of which you do not take issue with apparently). I get what you're saying, but I don't see how it contributes to either side of the debate.
Let me use this little quote to illustrate why I was frustrated with your first post. This quote clearly shows you are well able to understand the connection between use of benzodiazepines/antidepressants and anxiety/depression, which are symptoms of MDMA-abuse. Therefore, I find it strange that you respond to my argument with only one sentence, with which you obviously try to debunk my argument, while it also indicates you are not seeing the connection. Now when I read the above quote, it is more than clear you
do see the connection. Do you understand that this course of events leads me to believe you intentionally respond this way initially? To me, it comes off as a 'quick and dirty' way of discussing, the way crooked politicians usually do in public debates. And I emphasize: it comes off like this to me, I have no intention of flaming you, I am just trying to explain how I feel about your post.
As for the arguments in the quoted lines: I was under the impression it would be logical and/or known that selection bias virtually always undermines the conclusion of any study. Selection bias basically means that you are either putting
more of diseased/non-diseased subjects in one of the groups, or are putting
less of diseased/non-diseased subjects in one of the groups. The result is always the same: You find a larger or smaller effect size than the actual effect size, since the proportion in your diseased
or non-diseased group is larger/smaller than the actual proportion in the population.
Okay...haha...when reading back my explanation it looks like it is only making things harder, LOL. Think about this example, not necessarily comparable to the present study but the idea is the same: A study to the effects of smoking on cancer, where people that use anti-cancer drugs (which can also cause cancer) are excluded. This would mean that a lot of people that actually have (had) cancer, are excluded. Now, if smoking does
not cause cancer, this would not be a problem since the proportion of people with cancer would be the same in both groups. But since smoking
does cause cancer, more people in the smoking group will have (had) cancer, so more of them will have a history of taking cancer drugs, so more people with cancer will be excluded from the smoking than from the non-smoking group. The result is that you will find a smaller effect from smoking on cancer than there actually is.
Why should I appreciate a bad metaphor?
You don't have to. But it does not really add to the substance of the discussion to add it to your post either. Cracking up on good or bad metaphors is just an invitation for a flame war most of the time anyways...
.
