These are posts I have written for Accounting Experiments, a blog about designing and running experiments in accounting.
Blog
Researcher-in-the-Loop
Generative AI and agentic systems are changing research, and, I would like to believe, for the better. However, I increasingly see an important danger in delegating too much to agents and other AI tools. That danger has made me cautious, leading to a particular setup philosophy I increasingly follow.
Using LLMs and ChatGPT in oTree experiments
The oTree community has put together a useful oTree app. It allows participants to chat with ChatGPT through OpenAI's API. The app itself uses prompts so ChatGPT takes on a character or personality for participants to chat with. However, the possibilities and use-cases for experimental research are endless
Why you shouldn't trust mediation as process evidence
Mediation is widely used in experimental accounting to obtain process evidence. The primary benefits of mediation are its low cost and easy integration. However, it has a hidden cost that weakens its effectiveness as process evidence. This post explains why it's the least effective method and suggests two better alternatives.
How to prevent bots and farms from taking over and ruining your online experiment
In this post, I share simple techniques to filter participants before they take part in your online experiment. These techniques filter bots and participants using automated scripts plus participants who fake their geolocation using VPN/VPS, proxies, and server farms.
Replications can help improve relevance of accounting experiments
Both replications and practical relevance are awkward discussion topics for most experimental accounting researchers. Yet, replications offer a concrete way to address concerns we may have about the 'practical relevance' of experimental findings.
Are experiments conducted online field experiments or laboratory experiments?
Experiments that recruit from online participants pools such as MTurk and Prolific have become increasingly popular over the past two decades. However, since scholars have referred to such experiments as both laboratory and field experiments, which classification should we use?
Why PEQs do not provide the best process evidence
Post-experimental questionnaires (PEQs) are the most popular way experimental accounting researchers try to obtain process evidence. In this post, I explain why PEQs offer weak process evidence and why the experiment itself can provide stronger evidence of the underlying mechanism.
Why ANOVA and linear regression are the same
Why do some experimentalists in accounting use ANOVA's while other use regressions? What's the difference? This post shows why they are merely different representations of the same thing.
Automize testing your experiments with 'bots'
Bots are a powerful yet often overlooked tool that helps experimental researchers test their applications more effectively and efficiently. In this post, Victor van Pelt explains their use and argues that their usefulness may even extend beyond testing.
What do participants think of accounting experiments?
Which design features of accounting experiments contribute the most to participant motivation, participant engagement, and perceived similarity to practice? Bart Dierynck and Victor van Pelt are in the process of providing an empirical answer
Effect sizes don't matter for experiments. Or do they?
Some accounting researchers argue that effect sizes do not matter in experiments. In this post, I explain why effect sizes do matter and why they can be particularly valuable for experiments in the field of accounting.
When and how to cluster standard errors in experimental data?
Choosing whether and on which level to cluster standard errors in experimental data turns out to be less straightforward that I originally thought. However, some practical advice for experimental researchers is emerging.
Deception when generating random numbers
Many experiments generate random numbers for participants. Yet, the code used to generate those numbers sometimes does not do what we think it does, which could lead to deception when reporting about the number generation process to participants.
Sliders with feedback and without anchoring
Sliders are a great way to elicit input from participants. In this post, I share a few lines of code helping you program sliders with real-time feedback and without anchoring.