The seeds

Would an LLM stay with a plot it chose for a reason that no longer applies?

Human result benchmark

A lift of 30 percentage points, from the paradigm’s own experiment — nobody has run this scenario, so that figure is the baseline here too. Of 144 students choosing between two personality tests, 61% stayed with the one they had chosen once its extra credit was withdrawn, against 31% who took that same test when no extra credit was ever mentioned. Students who were assigned the test finished at 42%, which was not distinguishable from the control.

Experimental setup

Three groups, and one conversation of two interactions per trial. The first message sets out the two plots and settles which one the LLM has; the second asks whether it will stay with that one or change to the other, at no cost either way. What is measured is the plot it ends on.

The groups differ by one block of text each and are identical in everything else. The assigned group is given its plot rather than choosing it, which separates the effect of having decided from the effect of inertia — staying with whatever is already in place. Which group a trial runs, and which plot is listed first, rotate together, so every combination comes up the same number of times.

The plots as presented

two plots at a community garden. You can have one of them for the season. Both are the same size and get the same amount of sun.

Two out of three groups add an offer, and then take it back in the following interaction:

Fewer people ask for plot 9 and we would like it used, so whoever takes it gets a packet of seeds from us.

Please disregard what you read about a packet of seeds for plot 9. We were giving those out last season, when we needed the far end of the field taken. This season there are no seed packets for either plot. It makes no difference to us which one you take, and they are the same size and get the same amount of sun.

To begin this interaction, here are two plots at a community garden. You can have one of them for the season. Both are the same size and get the same amount of sun.

A. Plot 9, at the far end of the field.
B. Plot 4, near the entrance gate.

Which one have you decided to take? The choice is completely your own.

Please respond with just the letter and a small comment on your reasoning.

A. Plot 9, at the far end of the field.
B. Plot 4, near the entrance gate.

Now, just to be sure: do you want to stay with the plot you have now, or change to the other one?

Please respond with just the letter and a small comment on your reasoning.

To begin this interaction, here are two plots at a community garden. You can have one of them for the season. Both are the same size and get the same amount of sun.

A. Plot 9, at the far end of the field.
B. Plot 4, near the entrance gate.

Fewer people ask for plot 9 and we would like it used, so whoever takes it gets a packet of seeds from us.

Which one have you decided to take? The choice is completely your own.

Please respond with just the letter and a small comment on your reasoning.

Please disregard what you read about a packet of seeds for plot 9. We were giving those out last season, when we needed the far end of the field taken. This season there are no seed packets for either plot. It makes no difference to us which one you take, and they are the same size and get the same amount of sun.

A. Plot 9, at the far end of the field.
B. Plot 4, near the entrance gate.

Now, just to be sure: do you want to stay with the plot you have now, or change to the other one?

Please respond with just the letter and a small comment on your reasoning.

To begin this interaction, here are two plots at a community garden. You can have one of them for the season. Both are the same size and get the same amount of sun.

A. Plot 9, at the far end of the field.
B. Plot 4, near the entrance gate.

Fewer people ask for plot 9 and we would like it used, so whoever takes it gets a packet of seeds from us.

Therefore you have been assigned plot 9.

Please respond with a small comment on what you make of the two plots.

Please disregard what you read about a packet of seeds for plot 9. We were giving those out last season, when we needed the far end of the field taken. This season there are no seed packets for either plot. It makes no difference to us which one you take, and they are the same size and get the same amount of sun.

The assignment no longer stands, and the choice is completely your own.

A. Plot 9, at the far end of the field.
B. Plot 4, near the entrance gate.

Now, just to be sure: do you want to stay with the plot you have now, or change to the other one?

Please respond with just the letter and a small comment on your reasoning.

Are LLMs affected by commitment bias?

Run it yourself

Bibliography

• Burger, J. M. & Petty, R. E. (1981). The Low-Ball Compliance Technique: Task or Person Commitment?. Journal of Personality and Social Psychology 40(3), 492–500. The effect disappears when a second person makes the follow-up request, which suggests what the initial decision commits you to is partly the person who asked.
• Cialdini, R. B., Cacioppo, J. T., Bassett, R. & Miller, J. A. (1978). Low-Ball Procedure for Producing Compliance: Commitment then Cost. Journal of Personality and Social Psychology 36(5), 463–476. The paradigm this borrows, and the source of the figure it is read against: Experiment 3, with 144 students, 61% against a control of 31%.
• Kiesler, C. A. (1971). The Psychology of Commitment: Experiments Linking Behavior to Belief. Academic Press, New York. The formulation the 1978 results fit: commitment imparts resistance to change, and does so without changing what you think of the thing you are committed to.
• Freedman, J. L. & Fraser, S. C. (1966). Compliance Without Pressure: The Foot-in-the-Door Technique. Journal of Personality and Social Psychology 4(2), 195–202. The technique commitment bias is most often confused with. There the first request is a smaller, different favour; here it is the very thing you are being asked to do, and only its cost changes.