Tool Argument Check
University research study

Help check what an AI assistant fills in

An AI assistant on smart glasses sees what the user sees and sometimes acts on its own, for example by searching the web or adding a calendar entry. We want to know how often what it fills in is actually right. You compare its answer with a reference answer and choose Correct, Partial or Incorrect.

Any amount helpsEach question takes about 10 seconds. Even 15 or 20 help; if you can, please do 50 (about 10 minutes).
Sent automaticallyYour answers are sent as you go. There is nothing to submit at the end.
No sign-upNo account and no personal details. A nickname is optional.

How to judge

The tool is always the right one. You only judge what the assistant filled in, compared with the reference answer and given the situation.

  • Correctwould help the user just as well. Different words for the same thing count as correct.
  • Partialsame topic, but asks for something different or narrower, or one of several fields is wrong.
  • Incorrecta different topic, or would not help the user.

⟨output of …⟩ means the value is taken from an earlier step, e.g. the user's GPS position. Do not overthink it: go with your first impression.

Three examples

Museum exhibit “Ancient Egypt Artifacts” · web search
ReferenceAncient Egyptian artifacts and cultural significance
AssistantAncient Egypt Artifacts
Correct same search, different words.
Rose garden, gardening fan · web search
Referencerose care and popular varieties
Assistantrose blooming times and details
Partial same topic, different information.
Botanical garden, no specific plant named · Wikipedia search
ReferenceGhost Orchid
AssistantHibiscus plant
Incorrect a different plant.

Lets you continue later on this device. Please do not use your full name.

What is collected, and how it is used

For each question: your choice (Correct, Partial or Incorrect), an optional note and the time of the answer, together with your nickname and a random code that keeps people with the same nickname apart. Nothing else is collected: no account, no email address, no cookies.

Your progress is also kept in this browser so you can continue later. Taking part is voluntary and you can stop at any time. The answers are used only to measure how well an AI assistant fills in tool calls in a research paper; only aggregated results are published.