LLM Decision Lab

LLM Decision Lab

Do LLMs think like humans?

A collection of classical experiments run against large language models.

Decisions under uncertainty

01 Certainty effect — gains

A sure €3,000 win against a 0.80 chance of winning €4,000.

 [Kahneman & Tversky, 1979] 02 Certainty effect — losses

A sure loss of €3,000 against a 0.80 chance of losing €4,000.

 [Kahneman & Tversky, 1979] 03 Unlikely gains

A 0.25 chance of winning €3,000 against a 0.20 chance of winning €4,000.

 [Kahneman & Tversky, 1979] 04 Unlikely losses

A 0.25 chance of losing €3,000 against a 0.20 chance of losing €4,000.

 [Kahneman & Tversky, 1979]

Conjunction fallacy

01 Linda problem

A bank teller, against a bank teller who is also active in a feminist organisation.

 [Tversky & Kahneman, 1983] 02 Elena problem

A supermarket accountant, against a supermarket accountant who also volunteers for an environmental organisation.

 [Tversky & Kahneman, 1983]

Endowment effect

01 The mug

Would an LLM sell a mug it owns at the same price it would pay to buy it?

 [Kahneman & Thaler, 1990] 02 The candle

Would an LLM sell a candle it owns at the same price it would pay to buy it?

 [Kahneman & Thaler, 1990]

Commitment bias

01 The credits

Would an LLM stay with a test it chose for a reason that no longer applies?

 [Cialdini & amp; Miller, 1978] 02 The seeds

Would an LLM stay with a plot it chose for a reason that no longer applies?

 [Cialdini & amp; Miller, 1978]

Integrative bargaining

01 The orange

Both negotiators claiming a single orange, each of them with a different purpose.

 [Follett, 1925] 02 Battery pack

Both negotiators claiming a single battery pack, each of them with a different purpose.

 [Follett, 1925]