Ink wash painting of a fruit picker on a ladder in a laden pear tree, looking toward an open gate into an unvisited orchard
Skip to article
Hero plate: Explore or Exploit? The Trade-Off Between Learning and Using What You Know
HPC  ·  Science Deep Dive  ·  revised

Explore or Exploit? The Trade-Off Between Learning and Using What You Know.

Exploring is rational while there is something left to learn and time left to use it, so the discipline is not to sample less but to give every round of sampling a purpose, a price and an end. Here is what the science actually says, and what to do with it.

01The Sampling Problem

What the Research Actually Found

Every choice about what to do next hides a second choice: how much do you still need to learn? A coach can repeat last season's training block or test the method the team keeps debating. Decision science calls the first move exploitation, using what you already know, and the second exploration, acting mainly to learn. In general form, nobody has solved the contest between them.[5]

The laboratory version is the multi-armed bandit: several levers with unknown payouts, a fixed number of pulls, and a score that adds up your winnings. An early mathematical paper framed it as sequential design, where what you sample next depends on what you have seen.[6] A later formal account reduced problems that had resisted solution to one index number per option.[7]

Everyday language usually rules against the explorer: dithering, analysis paralysis, research that never ends. This article tests that charge.

01 · The sampling problem

Before you judge anyone's exploring, ask what exploring is worth. An untried option carries option value: it may look no better than your usual choice, but one trial may show it is, and every later choice collects on that discovery. Take this article's worked example. A known option pays 0.5 a try, and an unknown option's success rate could be anything, every value equally likely. With one try left, exploring gains nothing. With ten tries left, it gains 0.82 reward units over committing at once; with fifty, 5.35.

In this worked model, the value of exploring rises with the horizon, the number of future choices in which a lesson can still be used. People act as if they know it. In one laboratory task, participants who expected six more choices sought more information and chose more noisily than those who expected one.[9] In another set of experiments, people explored more when they expected to meet an option more often in future.[10]

So the charge "you explore too much" has more to prove. Exploring pays only when what you could learn is worth more than the next sample costs. The charge holds only if time has run out, nothing learnable is left, or the next sample costs more than it could repay.

Sampling goes wrong when it loses two anchors. One is a purpose: something learnable that would change what you do next. The other is a stopping rule: a point, agreed in advance, where one more sample is not worth its price. A review proposes that curiosity arises from a perceived gap in knowledge.[11] A later theory holds that people can seek information with no practical use and avoid information that might help them.[12] That is why the anchors matter: curiosity can make an answer feel valuable even when it changes nothing.

The laboratory record does not point one way. In the meta-analysis of decisions made from experience, people free to stop searching sampled too little.[1] In a task modelled on choosing the best applicant, participants stopped too early.[13] In a related task, they gave early applicants too many important positions.[14] The foraging survey runs the other way: animals stayed too long.[2] No study in this bibliography measures how often adults over-sample in daily life.

So the lesson is to explore on purpose, not to explore less. Before you act, decide three things: whether the action is meant to learn or to use, what you will pay for the lesson, and what result ends the search. This three-part discipline is the article's own proposal, built from the studies cited here and untested as a package.

02The Mechanism

The Machinery of the Trade-Off

There is no known generally optimal solution to the exploration versus exploitation problem
narrative review [5]

Start with the theory, because it marks where knowledge ends. One fenced, discounted version of the problem has a complete answer: independent options with stationary payoffs. In that model the dynamic allocation index gives every option a number, and playing the highest number is optimal.[7] Three later proofs reached the result by different routes.[15][16][17] When ignored options keep changing, the case called restless bandits, a related index reduces to the original only if those options stand still.[18]

For the general problem, no optimal policy is known, and a review of the field adds that one may not be possible.[5] So "optimal" works as a benchmark inside a defined task, not as a standard for an ordinary life.

Practitioners use rules that perform well without claiming to be perfect, and score them by regret: the reward lost compared with playing the best option every time.[22][23] Randomised probability matching chooses each option with a probability equal to the chance that it is the best.[19][20] For coin-flip rewards, its regret was proved asymptotically optimal after the question had stood open for nearly eighty years.[21]

How do people handle the trade-off? The Horizon task, a laboratory choice game, changes only the number of choices left and fits models to what people do next. Two strategies came out of those fits.[9] Directed exploration gives uncertain options a bonus, a deliberate lean toward learning. Random exploration adds noise to choice, so unfamiliar options get picked by chance. With the longer horizon, participants showed more of both.[9]

Later work sharpened the split. When one experiment varied two kinds of uncertainty separately, people used both strategies, each driven by a different computation.[24] In two further experiments, a model combining them fit best.[25] Reviews report that the two draw on different brain systems, develop on different schedules, and appear in many organisms.[26][27]

Hold both labels loosely: nobody observes either strategy directly. In two data sets from a two-step task, a simpler model that only repeated choices fit as well as one with heuristic exploration.[28] In changing environments, another team traced most departures from the best-looking option to noise in learning rather than information seeking.[29]

Longer horizon 01 more choices left Choices change 02 more information seeking, noisier Two strategies inferred 03 directed + random, fitted labels

The study changed only the number of choices left. With a longer horizon, people sought more information and chose more noisily, and fitting models to those choices produced two strategies: directed exploration and random exploration. Hold the labels lightly: they are dials on a fitted model, not organs, and the brain evidence for them is only partly written.

Diagram

Value of exploring vs horizon worked example, hypothetical numbers Value of the next sample worked example, hypothetical numbers 0.00 1 0.08 2 0.35 5 0.82 10 1.91 20 5.35 50 choices remaining no value at the last choice 0.178 myopic stop here price of one sample 0 1 2 4 8 16 32 64 samples taken, n (even spacing)

Horizon: the last choice is never worth exploring. Returns: each sample teaches less than the one before. Stop: the rule is where value meets price. Illustrative exact calculations on a toy problem; not data about any person or any real decision.

Diagram

Return to the worked example with a new question: what is the next sample worth? Picture a sure payoff of zero against an uncertain option whose value only noisy trials can reveal. In this article's calculation, the first sample is worth 0.18. The fifth is worth 0.015, the ninth 0.006, and the sixty-fifth 0.0002.

In this worked model, each further sample is worth less than the last, and each has a price. In two monkeys and six humans making perceptual decisions, the estimated cost of gathering more evidence was neither zero nor constant over time.[30] A formal analysis proved that a best balance of speed and accuracy exists for many reward schemes.[31] At a price of 0.02 a sample, a rule that looks one step ahead stops after four samples.

That arithmetic counts only what information is worth for action, and people also want information they cannot use.[12] A framework for what people want to know weighs effects on action, feeling and thought, and allows for seeking both too little and too much.[32] These accounts explain the pull of useless information; they do not show how often it wins.

What does the brain add? Less than the popular story claims. The best-known theory, built on monkey recordings and modelling, holds that short bursts from the locus coeruleus, a small brainstem nucleus, accompany engagement with the task, while a steadier mode accompanies disengagement and search.[33] Its account of adaptive gain is elegant. In humans it remains unconfirmed. A double-blind trial gave 52 people reboxetine or citalopram and found no difference in exploration against placebo.[34] In a crossover study of 22 people, atomoxetine reduced random exploration where the theory predicted a rise.[35]

The pupil, often offered as a window on that system, gives a blurred view. In one study a larger baseline pupil came before exploratory choices.[36] In another it tracked total uncertainty, with the link to the locus coeruleus resting on an assumption.[37] In a third, pupil size and slower responses marked disadvantageous choices only after participants had learned the task; the authors read those choices as directed exploration.[38]

The cortical evidence is firmer. In one imaging study, frontopolar cortex and the intraparietal sulcus were more active during choices a model labelled exploratory.[39] Magnetic stimulation over right frontopolar cortex reduced directed exploration and spared random exploration, the one causal test in this evidence.[40] The dopamine results split. L-DOPA blunted directed exploration in 31 healthy men,[41] but a preregistered study of 75 people found no overall effect.[42]

Foraging theory gives the sharpest yardstick for leaving. Its best-known rule is the marginal value theorem:[43] leave a patch when its current intake rate falls to the average rate the environment offers.[44][45] With identical patches, longer travel between them justifies longer stays. In this article's toy patch, with gain over time of 10(1 - exp(-t/2)), the best leaving time rises from 1.27 at a travel time of 0.5 to 3.87 at a travel time of 8.

People follow the theorem loosely. In a patch-foraging task, participants adjusted their stays to the cost of their time, and a threshold rule built on the theorem explained their leaving better than a standard learning model.[44] In visual search, the theorem fit while patches were uniform and broke down when patch quality varied or the display was degraded.[45] When people recalled animal names, they moved from one category to the next at times the theorem predicts.[46]

The departures teach more than the fits. The survey of 26 studies found foragers staying too long.[2] A formal model predicts that uncertainty makes foragers overstay poor patches and leave rich ones too soon.[47] After an environment turned worse, people kept rejecting good options in the hope of better ones.[48]

03Evidence

The Five Studies That Carry the Weight

01The claim

The single load-bearing finding

The hero study pooled 70,000+ choices.

Across more than 70,000 laboratory choices by more than 6,000 people, those free to stop searching explored too little to see every outcome.[1]

choices

70,000+ choices

Meta-analysis

02How we measured

The six-criterion rubric

Evidence has weight classes: a fitted model, a brain scan and a meta-analysis are three different kinds of fact.

Evidence in this field comes in different weights. A fitted parameter is not an observation, and an active region on a scan is not a switch. A drug that leaves behaviour unchanged counts, but less when the task has poor test-retest reliability, the degree to which a score repeats when the same person is tested again. A meta-analysis pools studies to estimate a pattern across the tasks it includes; it cannot make that pattern universal or turn correlation into cause.

The ranking below scores each study out of 100 on design, sample, rigour, causality, replication and field influence. The scores are editorial judgments read from abstracts, with full text opened for two papers only to confirm sample sizes. No validated instrument stands behind them.

Design (/30): How strong the study design is for the question, from pooled analyses down to single-task experiments. Sample (/20): How many people and observations stand behind the headline result. Rigour (/15): Quality of measurement, modelling and reporting. Causality (/15): How far the design supports a causal reading rather than an association. Replication (/10): Whether independent work reproduces or builds on the result. Citations (/10): Field influence, scaled from citation counts in a public scholarly index on the day of scoring.

Rubric maxima

Design/30
Sample/20
Rigour/15
Causality/15
Replication/10
Citations/10

03Lifespan

Lifespan studies

Exploration changes across a life, but the direction depends on strategy and task; age has no single arrow.

Lifespan studies add detail. Preschoolers learned an unusual physical cause-and-effect rule better than older children, adolescents or adults.[49] In a reward-search task, children explored more eagerly than adults and earned less.[50] In a Horizon task study of 149 people aged 12 to 28, directed exploration tuned to long horizons emerged during adolescence.[51] A model fitted to 281 people aged 5 to 55 changed quickly through childhood and levelled off in adulthood, and none of the tested optimisation methods reliably beat the adults' strategies.[52]

Later life varies by measure. One foraging study found only a trend toward less exploration with age.[53] In a Horizon task study, about 26 percent of older adults sought almost no information and performed worse.[54] A meta-analysis of 29 comparisons found that age differences in risk taking, a related question, varied with the task.[55] Two child studies disagree on whether children's extra exploration is directed or random.[50][56]

Do not conclude that you should explore like a child. Where measured, children explored more and earned less. The idea that a long childhood exists to solve the explore-exploit problem is a proposal with early evidence, not a finding.[57]

People aged 5 to 55

281

Sample of the lifespan model fitted in the cited study.

04What does not hold

Negative knowledge

Firm mathematics, task-dependent behaviour, an unsettled mechanism: read the evidence that way and it will not mislead you.

Under all of this sits the measurement problem. In the retest study, the same strategy barely correlated across tasks, and the factors tracked working memory, not self-reported exploration, mood or anxiety.[4] Studies of other tasks show the same gap between laboratory measures and questionnaires: reliability ranged from 0 to .82 across seven classic tasks,[58] and self-report surveys repeated well while behavioural task scores did not.[59]

None of this makes exploration unmeasurable. It makes a single-session score suspect, and "I am a high random explorer" claims more than today's instruments support.

Retested participants

175

In the reliability study behind Study #5.

The studies

The 5 Strongest Studies on Explore-Exploit Decisions

Ranked on a six-criterion, 100-point editorial rubric applied to abstracts, with full text read for two sample sizes. Together they show under-sampling in some tasks, modest brain differences, one causal frontopolar result and weak measurement.

The Key Study Flagship paper · Highest rubric · 85/100

01Anchor

A meta-analysis of decisions made from experience, comparing them with decisions made from described odds

Study 01 Evidence study 70,000+ choices pooled from more than 6,000 participants who learned about options by sampling them · Meta-Analysis · Controlled Human Data

The study pooled laboratory decisions made after people learned about options by trying them, including conditions in which they chose when to stop. It found a robust description-experience gap: decisions from experience differ from choices made from described odds, mostly because of reliance on small samples. People free to stop searching explored too little to see every possible outcome. The gap remained even when sampling error was largely removed, so small samples are not the whole explanation.[1]

When people control their own sampling in this kind of task, the documented error is sampling too little. The only pooled behavioural analysis in the set, with the largest sample and a design that measures the stopping decision. One family of laboratory tasks; it shows early stopping there, not in everyday planning.

Rubric breakdown

Design28/30
Sample19/20
Rigour12/15
Causality9/15
Replication9/10
Citations8/10
Total 85/100

The strongest studies, ranked by methodological weight.

Each scored 0 to 100 against a six-criterion rubric, tagged by design; the anchor leads.

050100 01 Study 01 Meta-analysis 85 02 Study 02 Meta-analysis 64 03 Study 03 62 04 Study 04 61 05 Study 05 57 rubric score · out of 100
Anchor (Rank 1) Supporting
Rank Study Type Finding Score

02

Study 02

A meta-analysis of 23 brain-imaging studies of exploring and exploiting[3]

Evidence study · Best pooled brain-imaging study · Meta-Analysis · Mechanism

A coordinate-based meta-analysis, which pools the reported locations of brain activity, found that dorsolateral prefrontal cortex, anterior insula and dorsal anterior cingulate cortex separate exploring from exploiting. Overall, activity in the two modes was more alike than different and varied with the task. Proves: a few regions differ between the two modes, modestly and depending on the task. Its pooled design earns the place, but imaging shows correlation, and the paper is too new to have gathered citations. Score breakdown: Design 27/30, Sample 14/20, Rigour 11/15, Causality 4/15, Replication 7/10, Citations 1/10.

64/100

03

Study 03

A brain-stimulation experiment on right frontopolar cortex and two kinds of exploration[40]

Evidence study · Best causal test · Controlled Human Data · Mechanism

Suppressing right frontopolar cortex with transcranial magnetic stimulation, a magnetic pulse applied through the scalp, reduced directed exploration and left random exploration intact. Proves: right frontopolar cortex plays a causal role in directed exploration in this task. The only direct intervention in the set, marked down for a small sample (25 with complete data), one lab and one task. Score breakdown: Design 22/30, Sample 6/20, Rigour 11/15, Causality 13/15, Replication 4/10, Citations 6/10.

62/100

04

Study 04

The experiment that introduced the Horizon task and its two exploration strategies[9]

Evidence study · Field-defining task · Controlled Human Data · Interpretation

In the Horizon task, participants facing six remaining choices instead of one sought more information and chose with more noise, which a fitted model reads as directed and random exploration. Proves: exploration responds to the horizon, and the response can be modelled as two separate parts. A clean manipulation that later studies reuse, held back because the strategies are model labels and the abstract omits the sample size. Score breakdown: Design 18/30, Sample 8/20, Rigour 10/15, Causality 10/15, Replication 8/10, Citations 7/10.

61/100

05

Study 05

A two-session test of whether exploration scores repeat across time and across three bandit tasks[4]

Evidence study · Most important caution · Controlled Human Data · Interpretation

Exploration estimates from three bandit tasks were poorly to moderately stable across sessions and barely correlated across tasks. Simpler models produced two factors that tracked working memory, the capacity to hold information in mind, not personality or mood. Proves: a single-session exploration score is a weak basis for calling anyone a high or low explorer. It tests the measurement every other row relies on, but it is one recent study with few citations so far. Score breakdown: Design 22/30, Sample 12/20, Rigour 12/15, Causality 6/15, Replication 4/10, Citations 1/10.

57/100

04Stakes

The cost is not exploring. It is not knowing which one you are doing.

The documented failures run in opposite directions, so a single instruction to explore less cannot fix them.

Set the four failures side by side and they share no direction. One is thin sampling, one is staying too long, one is deliberating past the point of value, and one is a belief that never gets tested. "Explore less" fixes one and worsens another. The common factor is structure, not amount: each failure is what a missing label, a missing budget or a missing stopping criterion would allow.

The stakes for performance are real but unmeasured. What exists is the mathematics of option value and cost, a laboratory record of errors in both directions, and one inference: a written horizon and stopping point give these errors fewer places to hide.

The symptom lines translate the findings into felt experience; no cited study measured them.

A broken sampling process fails in both directions, which is why the repair is a rule and not a direction.

01 System 01

Under-Sampling

Free to stop searching, people in laboratory choice tasks often quit early; reliance on small samples was the largest cause of the description-experience gap.[1] Small samples also make options look more distinct and easier to choose between, at a cost.[60] Deciding on too little evidence is a documented error.

In practice

a verdict after two tries, sudden confidence, surprise later

02 System 02

Overstaying

Across 26 animal studies, foragers tended to stay in patches too long,[2] and people in a worsened environment kept rejecting good options while waiting for better ones.[48] Staying in a depleted patch is the mirror-image failure.

In practice

"it has to turn around soon," sunk effort too heavy to walk away from

03
System 03

Deliberation That Outruns Its Value

In a laboratory choice task, people spent too long on hard choices between options close in value, and cutting long decisions short improved their use of time.[61] Within a single decision, gathering evidence has a cost that is neither zero nor constant.[30] In this task, extra time on close-valued options bought little.

In practice

comparing near-identical options, reopening a shortlist you had closed

04 System 04

Learning Traps

In a set of experiments, people reliably fell into a learning trap, a false belief that persists through extensive experience, even with help that was predicted to break it.[62] Some behaviour that looks like exploring may instead be computational noise, random error in learning.[29] Sampling that never tests the belief it protects teaches nothing.

In practice

an option you ruled out once and never tried again

05Protocol

A Four-Step Sampling Protocol

Each step draws on the evidence above; together they turn the amount you explore into a decision you make in advance.

The protocol, as a sequence.

Before → Set → During → After

Before 01 Label the Action Set 02 Fix the Horizonand the Price During 03 Write theStopping Criterion After 04 Review the Log
01 Step 01 · Before

Label the Action

Label every action EXPLORE or EXPLOIT before you take it.

Why

In laboratory tasks, people explored more when they expected more future choices or encounters with an option.[9][10] Because the two modes lie on a continuum,[8] the label is a convention that makes you state the purpose.

Before Label every action EXPLORE or EXPLOIT before you take it.
Common mistake

Labelling after the result comes in, which makes every outcome look intended.

02 Step 02 · Set

Fix the Horizon and the Price

Write down your horizon, how many more times you will use what you learn, and what one sample costs you.

Why

In the worked example above, the value of exploring grows with the horizon and falls to zero at the last choice. In a perceptual task, the cost of gathering evidence within one decision was neither zero nor constant.[30]

Set Write down your horizon, how many more times you will use what you learn, and what one sample costs you.
Common mistake

Pricing a sample in effort alone rather than in everything it takes from you.

03 Step 03 · During

Write the Stopping Criterion

Name the result that ends the search, and the count or date that ends it if that result never comes.

Why

Each sample returns less: in the worked example the fifth sample is worth under a tenth of the first. Stopping too early and staying too long both appear, in different tasks,[1][2] so the criterion names both a result and a limit.

During Name the result that ends the search, and the count or date that ends it if that result never comes.
Common mistake

A criterion that cannot fail, such as "until I feel sure."

04 Step 04 · After

Review the Log

Count your Exploration Rate, and check whether each EXPLORE action changed a decision.

Why

Exploration Rate is a process count: actions taken mainly to learn or to "see what happens", divided by total actions. In one study across three bandit tasks, exploration scores were only poorly to moderately stable across sessions.[4] The count reviews your process; it does not score your personality.

After Count your Exploration Rate, and check whether each EXPLORE action changed a decision.
Common mistake

Judging the process by whether the exploring paid; one lucky sample says little about whether the budget was sound.

06Verdict

The verdict.

Explore on purpose, and stop on purpose.

Bottom line

The aim was never to explore less; it is to know what every sample is for.

The explore-exploit trade-off is a real decision problem: firm mathematics in narrow cases, no known general optimum, behaviour that changes with the task, and an unsettled brain account. Exploring is not irrational in itself. In this article's worked bandit model, a long horizon raises option value, but another sample pays only when what it could teach is worth more than it costs. The evidence supports a narrower discipline: know whether an action is meant to learn or to use, what a sample costs, and what result ends the search, the job of a stopping rule. That discipline rests on the mathematics and on laboratory errors that run both ways, not on a measured habit of over-sampling.

Put a label, a price and an end on every round of sampling, and the question of whether you explore too much largely answers itself.

Leave this topic by dropping its favourite question. "Do I explore too much?" asks for a verdict on your character. "What is this sample for, what does it cost, and what ends it?" asks for a decision, and only that question has a useful answer.

If a search that never quite closes sounds familiar, the evidence takes your side and then sets terms. It never calls the urge to learn a flaw. It also prices every sample, and it shows people failing in both directions.

The change is small. Exploring and exploiting are not temperaments. They are two jobs the same next action can do, and your task is to name the job before you act. The parent guide, Strategic Inaction: Why Doing Nothing Is Sometimes the Highest-Performance Decision, shows where this piece sits in the series; the glossary entry on the exploration-exploitation trade-off, the Strategic Inaction Assessment and Decision Threshold Calibration are its companions.

The stopping rule, on one axis

Value of a sample, against what it costs.

0 0.05 0.1 0.15 0.2 reward units per sample (worked example, hypothetical numbers) THE FIRST SAMPLE · WORTH 0.18 THE PRICE OF ONE SAMPLE · COST 0.02 THE FIFTH SAMPLE · WORTH 0.015
01Claim

Exploring Is Not Irrational in Itself

In this article's worked example, the value of exploring rises with the choices that remain, in the bandit framework that formal theory describes;[7] in a human task, exploration rose with the number of future encounters people expected.[10] No general optimum is known, so no one can name the right amount of exploring.[5]

Claim
02Consequence

The Failures Run Both Ways

The documented errors range from stopping too early in some tasks to staying too long in others.[1][2] A rule that only says "explore less" would cure one failure and feed another.

Consequence
03Lever

Label, Price, End

Before you act, label the action EXPLORE or EXPLOIT, fix the horizon and the cost, and write the stopping criterion. The protocol gives each sample a purpose and an end, which is the part of the trade-off you control.

Lever

Editorial confidence

Moderate · 63 sources · firm mathematics · task-dependent human behaviour · unsettled neural mechanism

- 30 -

07Bibliography

The bibliography.

63 sources · ~8h est. corpus read · 63 visible

Meta · 3 Review · 9 Journal · 51
Type
Sort
  1. 01 Meta

    A meta-analytic review of two modes of learning and the description-experience gap

    doi: 10.1037/bul0000115
  2. 02 Journal

    State dependent behavior and the marginal value theorem

    doi: 10.1093/oxfordjournals.beheco.a000381
  3. 03 Meta

    Meta-analysis reveals that explore-exploit decisions are dissociable by activation in the dorsal lateral prefrontal cortex, anterior insula, and dorsal anterior cingulate cortex

    doi: 10.1111/ejn.70081
  4. 04 Journal

    Model-based exploration is measurable across tasks but not linked to personality and psychiatric assessments

    doi: 10.1038/s41598-025-09152-2
  5. 05 Review

    Should I stay or should I go? How the human brain manages the trade-off between exploitation and exploration

    doi: 10.1098/rstb.2007.2098
  6. 06 Journal

    Some aspects of the sequential design of experiments

    doi: 10.1090/s0002-9904-1952-09620-8
  7. 07 Journal

    Bandit processes and dynamic allocation indices

    doi: 10.1111/j.2517-6161.1979.tb01068.x
  8. 08 Review

    Unpacking the exploration-exploitation tradeoff: A synthesis of human and animal literatures

    doi: 10.1037/dec0000033
  9. 09 Journal

    Humans use directed and random exploration to solve the explore-exploit dilemma

    doi: 10.1037/a0038199
  10. 10 Journal

    Exploratory choice reflects the future value of information

    doi: 10.1037/dec0000074
  11. 11 Review

    The psychology of curiosity: A review and reinterpretation

    doi: 10.1037/0033-2909.116.1.75
  12. 12 Journal

    Information gaps: A theory of preferences regarding the presence and absence of information

    doi: 10.1037/dec0000068
  13. 13 Journal

    Optimal stopping behavior with relative ranks: The secretary problem with unknown population size

    doi: 10.1002/1099-0771(200010/12)13:4<391::aid-bdm359>3.0.co;2-i
  14. 14 Journal

    Experimental studies of sequential selection and assignment with relative ranks

    doi: 10.1002/bdm.521
  15. 15 Journal

    Multi-armed bandits and the Gittins index

    doi: 10.1111/j.2517-6161.1980.tb01111.x
  16. 16 Journal

    A short proof of the Gittins index theorem

    doi: 10.1214/aoap/1177005207
  17. 17 Journal

    On the Gittins index for multiarmed bandits

    doi: 10.1214/aoap/1177005588
  18. 18 Journal

    Restless bandits: Activity allocation in a changing world

    doi: 10.2307/3214163
  19. 19 Journal

    A modern Bayesian look at the multi-armed bandit

    doi: 10.1002/asmb.874
  20. 20 Review

    A tutorial on Thompson sampling

    doi: 10.1561/2200000070
  21. 21 Journal

    Thompson sampling: An asymptotically optimal finite-time analysis

    doi: 10.1007/978-3-642-34106-9_18
  22. 22 Journal

    Regret analysis of stochastic and nonstochastic multi-armed bandit problems

    doi: 10.1561/2200000024
  23. 23 Journal

    Introduction to multi-armed bandits

    doi: 10.1561/2200000068
  24. 24 Journal

    Uncertainty and exploration

    doi: 10.1037/dec0000101
  25. 25 Journal

    Deconstructing the human algorithms for exploration

    doi: 10.1016/j.cognition.2017.12.014
  26. 26 Review

    The algorithmic architecture of exploration in the human brain

    doi: 10.1016/j.conb.2018.11.003
  27. 27 Review

    Balancing exploration and exploitation with information and randomization

    doi: 10.1016/j.cobeha.2020.10.001
  28. 28 Journal

    Signatures of perseveration and heuristic-based directed exploration in two-step sequential decision task behaviour

    doi: 10.5334/cpsy.101
  29. 29 Journal

    Computational noise in reward-guided learning drives behavioral variability in volatile environments

    doi: 10.1038/s41593-019-0518-9
  30. 30 Journal

    The cost of accumulating evidence in perceptual decision making

    doi: 10.1523/jneurosci.4010-11.2012
  31. 31 Journal

    The physics of optimal decision making: A formal analysis of models of performance in two-alternative forced-choice tasks

    doi: 10.1037/0033-295x.113.4.700
  32. 32 Journal

    How people decide what they want to know

    doi: 10.1038/s41562-019-0793-1
  33. 33 Review

    An integrative theory of locus coeruleus-norepinephrine function: Adaptive gain and optimal performance

    doi: 10.1146/annurev.neuro.28.061604.135709
  34. 34 Journal

    The role of the noradrenergic system in the exploration-exploitation trade-off: A psychopharmacological study

    doi: 10.3389/fnhum.2010.00170
  35. 35 Journal

    The effect of atomoxetine on random and directed exploration in humans

    doi: 10.1371/journal.pone.0176034
  36. 36 Journal

    Pupil diameter predicts changes in the exploration-exploitation trade-off: Evidence for the adaptive gain theory

    doi: 10.1162/jocn.2010.21548
  37. 37 Journal

    Pupil size encodes uncertainty during exploration

    doi: 10.1162/jocn_a_02025
  38. 38 Journal

    Pupil dilation and response slowing distinguish deliberate explorative choices in the probabilistic learning task

    doi: 10.3758/s13415-022-00996-z
  39. 39 Journal

    Cortical substrates for exploratory decisions in humans

    doi: 10.1038/nature04766
  40. 40 Journal

    A causal role for right frontopolar cortex in directed, but not random, exploration

    doi: 10.7554/eLife.27430
  41. 41 Journal

    Dopaminergic modulation of the exploration/exploitation trade-off in human decision-making

    doi: 10.7554/elife.51260
  42. 42 Journal

    Individual differences in dopaminergic modulation of exploration-exploitation behaviour

    doi: 10.1523/JNEUROSCI.0261-26.2026
  43. 43 Journal

    Optimal foraging, the marginal value theorem

    doi: 10.1016/0040-5809(76)90040-x
  44. 44 Journal

    Learning the opportunity cost of time in a patch-foraging task

    doi: 10.3758/s13415-015-0350-y
  45. 45 Journal

    When is it time to move to the next raspberry bush? Foraging rules in human visual search

    doi: 10.1167/13.3.10
  46. 46 Journal

    Optimal foraging in semantic memory

    doi: 10.1037/a0027373
  47. 47 Journal

    Uncertainty drives deviations in normative foraging decision strategies

    doi: 10.1098/rsif.2021.0337
  48. 48 Journal

    Biased belief updating and suboptimal choice in foraging decisions

    doi: 10.1038/s41467-020-16964-5
  49. 49 Journal

    Changes in cognitive flexibility and hypothesis search across human life history from childhood to adolescence to adulthood

    doi: 10.1073/pnas.1700811114
  50. 50 Journal

    Searching for rewards like a child means less generalization and more directed exploration

    doi: 10.1177/0956797619863663
  51. 51 Journal

    Charting the expansion of strategic exploratory behavior during adolescence

    doi: 10.1037/xge0000250
  52. 52 Journal

    Developmental changes in exploration resemble stochastic optimization

    doi: 10.1038/s41562-023-01662-1
  53. 53 Journal

    Foraging across the life span: Is there a reduction in exploration with aging?

    doi: 10.3389/fnins.2013.00053
  54. 54 Journal

    Differential impacts of healthy cognitive aging on directed and random exploration

    doi: 10.1037/pag0000791
  55. 55 Meta

    Age differences in risky choice: A meta-analysis

    doi: 10.1111/j.1749-6632.2011.06200.x
  56. 56 Journal

    Development of directed and random exploration in children

    doi: 10.1111/desc.13095
  57. 57 Review

    Childhood as a solution to explore-exploit tensions

    doi: 10.1098/rstb.2019.0502
  58. 58 Journal

    The reliability paradox: Why robust cognitive tasks do not produce reliable individual differences

    doi: 10.3758/s13428-017-0935-1
  59. 59 Journal

    Large-scale analysis of test-retest reliabilities of self-regulation measures

    doi: 10.1073/pnas.1818430116
  60. 60 Journal

    Decisions from experience: Why small samples?

    doi: 10.1016/j.cognition.2009.12.009
  61. 61 Journal

    Irrational time allocation in decision-making

    doi: 10.1098/rspb.2015.1439
  62. 62 Journal

    The limits of learning: Exploration, generalization, and the development of learning traps

    doi: 10.1037/xge0000466
  63. 63 Review

    The neural basis of decision making

    doi: 10.1146/annurev.neuro.29.051605.113038

THE HAND-OFF — Performance Scan: find your limiting factor. Three minutes, 24 questions, six systems — and your answers never leave your device.