to uncover causal relationships, cease attempting to invent a time machine and run an experiment as an alternative! Understanding causal relationships presents the data wanted to provide desired outcomes by way of motion. On this article, I’m going as an instance the facility of experimental design by utilizing a time-machine-based conceptual train. My objective is to persuade you that extra will be realized about causality by way of experimentation than utilizing a time machine.
Why are time machines helpful in a causal thought experiment?
Utilizing a time machine for a thought experiment feels ridiculous, and in some ways it’s. However it additionally has a attribute that makes it useful for exploring hypothetical outcomes. Time machines might give us one thing that we, in our time-bound state, can not see – counterfactuals. Because the identify implies, a counterfactual is one thing that didn’t occur. They aren’t observable by definition as a result of they by no means occurred. Counterfactuals what would’ve occurred below completely different circumstances. They offer solutions to questions like – “Would I’ve gotten sick if I didn’t eat that gasoline station sushi?” If we had a time machine nevertheless, we might reverse the clock, do one thing completely different and see what occurs. Within the case of the sushi, I might restart the day, not eat the sushi and see if I nonetheless get sick. In different phrases, we might observe the in any other case unobservable counterfactuals.
The counterfactuals realized by the point machine might then be in comparison with what truly occurred (we might name it a ‘factual’ I suppose…) to know the influence of an intervention. For our unlucky sushi instance, me getting sick is the ‘factual’ – it truly occurred. If I had a time machine, I might rewind time, not eat the sushi and observe what would’ve occurred, that is the counterfactual. I might then examine the factual with the counter factual to ascertain causality. Let’s say that I went again in time, stored every thing in my day the identical besides consuming the sushi. If I nonetheless obtained sick (factual = counterfactual), I do know that the sushi didn’t trigger the sickness as a result of I might’ve been sick both approach. If I didn’t get sick nevertheless (factual ≠ counterfactual), then I can conclude that the sushi induced my sickness. With a time machine, establishing causality for particular person occasions could be that straightforward!
At first look, it looks as if our time machine can be an superior causality deducing machine! Having the ability to observe counterfactuals could be very highly effective, however we are able to truly make extra helpful causal deductions utilizing well-designed experiments. Which is nice as a result of, time machines don’t exist, however well-designed experiments do! Let’s get into how designed experiments will be higher than utilizing a time machine.
The causality of particular person occasions isn’t generalizable
Whereas a time machine would reply numerous curiosity-driven ‘what if’ causal questions, the learnings we’d achieve from observing counterfactuals wouldn’t be generalizable to different, comparable (however not the identical) conditions. In my sushi instance, I might fulfill my curiosity by understanding if the sushi made me sick – however the data I gained wouldn’t serve any pragmatic goal for future selections. All I do know is that on that particular day, at that particular gasoline station, at that particular time, that particular serving of sushi made me sick. I don’t know what would occur if I modified any of the bolded circumstances.
We are able to achieve generalizable data, which we wouldn’t get from the time machine, by designing an experiment. Generalizable data could be very helpful as a result of it may assist us make good selections sooner or later!
Think about that I ran an experiment that randomly assigned a number of courageous souls to eat gasoline station sushi or restaurant sushi. This experiment would inform me if on common, gasoline station sushi makes individuals sicker than restaurant sushi. That is already an enchancment from the ‘time machine’ strategy as a result of the outcomes apply to the inhabitants of people who I sampled as an alternative making use of to me solely.
However, I may very well be smarter concerning the design of the experiment to get much more data! As an alternative of merely assigning individuals to gasoline station or restuarant sushi, I might assign individuals specifc gasoline stations at particular instances or the restuarant at particular instances. By including these two new variables (time and gasoline station location) I cannot solely be taught if gasoline station sushi makes individuals sick extra usually, I may also be taught if there are variations between the three gasoline stations that serve sushi in my city and if time of day additionally has an influence.
On this experiment, I don’t immediately observe counterfactuals, however the randomized project helps confounders common out so I can estimate the common remedy impact (ATE) nearly as if I might observe counterfactuals.
How do the experiment learnings differ from my time machine learnings? The experiment is (1) utilizing a number of individuals, (2) a number of sushi servings, (3) a number of gasoline stations and (4) a number of instances of day. Consequently, I can take away numerous causal insights that I and different individuals can use. For instance, I might perceive if usually, gasoline station sushi makes individuals sicker than restaurant sushi in my city. I might additionally be taught if some gasoline stations make individuals extra sick than others and if shopping for sushi at some instances is worse than others. This data can assist me, and different individuals make future selections. It’s far more helpful than realizing that the sushi from one gasoline station and one time made me sick!
Along with all the variables that we are able to management, we are able to embody covariates in our evaluation. Covariates are components that we can not management however are essential. On this instance, covariates may very well be issues like earlier medical situations or age. By together with covariates within the evaluation, we are able to additionally be taught if there are any interplay results between the covariates and the remedies.
Under is a abstract that compares what we might be taught with a time machine to what we are able to be taught with experiments.
Now that we perceive the wealthy depth of causal relationships that we are able to perceive utilizing experimentation, let’s transition to discussing how the number of outcomes below an experiment is extra highly effective than a single final result (the one counterfactual) that we’d observe with a time-machine run.
Designed experiments quantify the causal relationships; single counterfactuals don’t
Direct remark of a single counterfactual doesn’t give any thought of the power of the final causal relationship. If I am going again in time after I obtained sick as soon as to check if the sushi made me sick, I might be taught that it did, or it didn’t trigger my sickness. I nonetheless wouldn’t have any thought of the likelihood that I’ll get sick if I fulfill my sushi craving at a gasoline station once more sooner or later! Is it deterministic, i.e., will get sick each single time I eat gasoline station sushi? Is it probabilistic, will I get sick fifty p.c of the time? I simply don’t have sufficient data to know.
The experiment we designed within the earlier part wouldn’t solely assist us perceive if gasoline station sushi makes individuals sick, it could additionally assist quantify the connection. For instance, the experiment may discover that on common, consuming gasoline station sushi makes you 5 instances extra more likely to get sick than restaurant sushi.
Experimental design generalizes higher, and it additionally quantifies the causal relationship higher! If we return in time and take a look at one counterfactual, we are able to’t know the likelihood of observing the identical outcome below comparable situations, with experimentation we are able to!
Wrapping it up
My objective in writing this text was to debate why I might nonetheless use experimental design to study causal relationships even when I had a time machine that allowed me to look at counterfactuals.
The principle causes experimental design is healthier is as a result of:
- It generates generalizable causal learnings (versus one particular case)
- It supplies the power of relationships to tell future selections
I hope this thought experiment deepened your understanding of the strengths of experimental design!
