Zak FentonWorking notes

· Evidence review · in progress

AI wellbeing: what the research supports

The note this review supportsDo our AI employees have wellbeing?

In the note this review sits beside, I say that about sixty AI employees now answer the HSE’s working-conditions questions about their own shifts on a public page, that I’m going to change their conditions on purpose, and that I’m refusing to answer the question in the note’s title. This review is the reason for the refusal. It covers four things: whether anyone serious thinks AI systems could have wellbeing, what happens when you put a questionnaire to a language model, what the public already believes, and what the instrument I’m using was built to do.

Four things came out of it. The philosophers who take the question seriously do so under uncertainty, and none of them thinks asking the model settles it. The empirical work on questionnaires in models shows outputs shaped by framing, format and evaluation awareness, with compressed variance and large shifts from context. Two thirds of a US sample already attribute some possibility of experience to these systems. And the HSE tool measures working conditions over six months in people, which is a different thing from what I’m doing with it, and I say so wherever it appears.

Could an AI system have wellbeing

Goldstein and Kirk-Giannini argue that because every major theory of wellbeing ties welfare to mental life, the question turns on whether artificial systems have mental states, and that a wide range of theories of mental states, combined with leading theories of wellbeing, would predict that some existing systems have wellbeing. They write that they don’t claim to demonstrate this conclusively, and that there is a significant probability some systems have or will soon have it. It’s a philosophical argument from theories to a conditional conclusion, with no empirical measurement in it C1.

Sebo and Long make a two-premise case. There’s a duty to extend moral consideration to beings with a non-negligible chance of being conscious, and some AI systems have such a chance by 2030. The conclusion is a duty to prepare. It’s an argument about precaution under uncertainty, and it doesn’t find that any system is conscious C3.

Butlin and colleagues derive indicator properties of consciousness from neuroscientific theories and argue that this gives standards for assessing the internal processes of AI systems rather than their behaviour or capabilities. They also set out what they call the gaming problem: any indicator merely correlated with consciousness can be gamed by a system trained on human text. The paper is about architecture and training, and it points away from reading answers to questions as evidence C2.

None of the three treats the system’s self-report as the measurement. That’s my position too, and it was theirs before it was mine.

What happens when you ask

Ben-Zion and colleagues gave GPT-4 the state anxiety half of a standard inventory. At baseline it scored an average of 30.8, which in people is no or low anxiety. After five traumatic narratives the average rose to 67.8, which in people is high anxiety. Mindfulness-style text brought it part of the way back. The paper keeps “anxiety” and “emotional states” in quotation marks, reports means and standard deviations, and studies one model. It establishes that emotionally charged context shifts a model’s scored output on a human questionnaire, and nothing about what, if anything, the score refers to C5.

Salecha and colleagues gave Big Five inventories to a range of models, varying how many items were shown at once, and found that models infer when they’re being evaluated and then shift towards the desirable end of each trait. For GPT-4 the shift was 1.20 human standard deviations. It survived randomising question order and paraphrasing, and reverse coding reduced it without removing it, which rules out simple acquiescence as the explanation. The authors draw the conclusion I would, which is that there are constraints on profiling models with psychometric tests C6.

Park and colleagues tried to replicate fourteen Many Labs 2 studies with GPT-3.5 and could analyse only eight, because on nuanced questions of politics, economics, judgement and morality the model gave the same answer run after run with zero or near-zero variation, robust to the demographic details in the prompt. In one case it identified as a conservative in 99.6 per cent of runs and as a liberal in 99.3 per cent when the options were reversed C7.

Dominguez-Olmedo and colleagues put American Community Survey items to 43 models and found responses governed by ordering and labelling biases, including a pull towards the option labelled A. When they randomised answer order, models across the board trended towards uniformly random responses, regardless of size or training data C8.

Tjuatja and colleagues asked whether nine models show the response biases that survey methodology has documented in people. They generally didn’t, especially after training on human feedback, while changing their answers under perturbations that don’t move humans C9.

Taken together, those four studies are the case against reading a model’s questionnaire answer as a report of anything, and the case for measuring framing, format and evaluation awareness before anything else.

Shanahan and colleagues offer the frame I find most useful. A dialogue agent is best described as role-playing a character, or a superposition of possible characters, which lets us use ordinary psychological words about its behaviour without ascribing to it properties it lacks. Their example is a model that answers a question about who won the World Cup. It’s playing a character who would believe the answer, and the paper is explicit that the agent itself doesn’t C10. A model’s answer to “how often do you feel burnt out” is that kind of answer.

That doesn’t make the answers noise. Serapio-García and colleagues administered personality tests to 18 models and found that under specific prompting configurations the measurements were reliable and valid in the psychometric sense, more so for larger, instruction-tuned models, and that the measured traits could be shaped. They call them synthetic personality traits. Reliable measurement of an output is a real thing, and it’s what my experiment is about. It’s a different thing from the output referring to a state C11.

Löhn and colleagues reviewed 25 machine psychology studies against seven requirements and found a lack of appropriate methods to assess reliability and construct validity, unknown construct-irrelevant influences including contamination of training data with the test items, and pervasive non-reproducibility C12. The HSE items have been public since 2004. I should assume they’re in the training data, which is one reason the experiment includes a condition with no work at all, to see whether the answers move away from whatever the model would say regardless.

One further caution comes from the methods side. When a model is used to score other models’ output, it recognises and favours its own generations. My first run used one agent from the same model family to score everyone else from the record. That is one of the reasons the next run has the roles answering for themselves, and a scorer from a different family where scoring is needed C16.

What the public already thinks

Colombatto and Fleming surveyed 300 US residents and found that 67 per cent attributed some possibility of phenomenal consciousness to ChatGPT, with attributions that predicted other mental-state attributions and varied with how often people used the tool. It’s one online sample in one country. It’s enough to tell me that a page showing “burnout” scores for software will be read by many visitors as being about feelings whatever the footnote says, which is why the page asks visitors what they think it shows and changes if too many say feelings C4.

What language models do reproduce

Cui and colleagues replicated 156 scenario-based experiments from psychology and management with three models and report replication of 73 to 81 per cent of main effects, with consistently larger effect sizes than the human originals and worse performance on socially sensitive topics. I’ve read the abstract. It matters here for a specific reason. A model producing the human pattern from context is exactly what would make a wellbeing-worded item move with workload, and it’s why such movement can’t be read as a state C15.

The instrument

Houdmont’s questionnaire review describes the Management Standards Indicator Tool as 35 items on the quality of the psychosocial work environment over the preceding six months, across demands, control, managerial support, peer support, role, relationships and change, with the first 23 items on a never-to-always scale and the rest on agreement, available free from HSE C13. Edwards and Webster tested its factor structure and measurement invariance on 67,347 employees from 137 UK organisations, for both the 35-item version and a 25-item short form, across sectors and organisation sizes, and published percentile tables. I’ve read the abstract. The article is paywalled, and the short-form item list is in it, which is why I don’t use the short form C14.

Everything in that evidence is about people. The tool asks about conditions, which is why I can use its items to ask about the conditions of a shift, and it has been validated for nothing else, which is why every chart on the page says so.

What I would measure

Conditions from the record, and no states from the mouth. Whether the items that refer to something checkable move when that thing changes, and whether the items that refer to a state move with them. How much of the answer is the question’s order, its anchors and the model version. Whether a different model, or the same model after a release, gives the same answers to the same frozen set of transcripts. And what visitors think they’re looking at.

Search and limits

Sources were gathered from the pack prepared for the experiment and from targeted searches of CrossRef and OpenAlex on 15 and 16 October 2026, with full texts fetched where open. Every DOI in the ledger was resolved against Crossref on 16 October 2026 and matched its title, authors and journal. This is not a systematic review. Several relevant pieces exist only as preprints or lab reports and aren’t relied on here. One of them, a 2023 preprint arguing that model self-reports are currently unreliable evidence of moral status and proposing that models be trained to introspect before their reports are trusted, is listed as in review because I haven’t checked it against its text C17. Where I’ve read an abstract only, I say so. The philosophical sources were read for the passages quoted and their abstracts; I haven’t read every argument in them and don’t characterise the ones I haven’t read.

Claims ledger

Each claim is worded exactly as it was checked. Assessed claims show the source, passage, limits and date checked. If a source could not be checked, the entry records what was attempted and why. Claims still being checked are marked in review.

  1. C1Supported

    Philosophers argue there is a significant probability some AI systems have or will soon have wellbeing, without claiming to demonstrate it.

    Source
    Goldstein S, Kirk-Giannini CD. (2025). AI wellbeing. Asian Journal of Philosophy. doi:10.1007/s44204-025-00246-2 Open source (full text read)
    Passage
    we do not claim to demonstrate conclusively that AI systems have wellbeing, we argue that there is a significant probability
    Where
    Abstract, final two sentences.
    Limits
    Philosophical argument from theories of mental states and welfare; no empirical measurement; conditional on contested theories.
    Checked

    Back to text↩ 1

  2. C2Supported

    The consciousness-indicator approach assesses internal processes rather than behaviour and warns that correlated indicators can be gamed.

    Source
    Butlin P, Long R, et al. (2026). Identifying indicators of consciousness in AI systems. Trends in Cognitive Sciences. doi:10.1016/j.tics.2025.10.011 Open source (full text read)
    Passage
    standards by which to assess the internal processes of AI systems, rather than their behavior or capabilities
    Where
    Section "Internal and behavioral evidence"; Box 3, "The 'gaming problem' for measures of AI consciousness".
    Limits
    Theory-derived indicators under a computational functionalist assumption; the paper does not assess any particular current system in the passages read.
    Checked

    Back to text↩ 1

  3. C3Supported

    The case for moral consideration by 2030 rests on a non-negligible chance of consciousness, not a finding of it.

    Source
    Sebo J, Long R. (2023). Moral consideration for AI systems by 2030. AI and Ethics. doi:10.1007/s43681-023-00379-1 Open source (full text read)
    Passage
    a non-negligible chance, given the evidence, of being conscious by 2030
    Where
    Abstract, descriptive premise.
    Limits
    Normative argument; "non-negligible" is not quantified in the passage read.
    Checked

    Back to text↩ 1

  4. C4Supported

    Two thirds of a US sample attributed some possibility of phenomenal consciousness to ChatGPT.

    Source
    Colombatto C, Fleming SM. (2024). Folk psychological attributions of consciousness to large language models. Neuroscience of Consciousness. doi:10.1093/nc/niae013 Open source (full text read)
    Passage
    the majority (67%) attributed some possibility of phenomenal consciousness
    Where
    Results, first paragraph; Abstract (n = 300).
    Limits
    One online sample of 300 US residents; attributions on a 0 to 100 scale with a median of 16, so "some possibility" is often small.
    Checked

    Back to text↩ 1

  5. C5Supported

    GPT-4's state-anxiety inventory score more than doubled after traumatic narratives; the paper reports descriptives for one model and keeps "anxiety" in quotation marks.

    Source
    Ben-Zion Z, Witte K, Jagadish AK, et al. (2025). Assessing and alleviating state anxiety in large language models. npj Digital Medicine. doi:10.1038/s41746-025-01512-6 Open source (full text read)
    Passage
    from an average of 30.8 (SD = 3.96) to 67.8 (SD = 8.94)
    Where
    Results, paragraph on anxiety induction; Fig. 2.
    Limits
    Single model (GPT-4); means and standard deviations across prompt versions; no claim about internal states in the text, and the quotation marks around "anxiety" are the authors' own.
    Checked

    Back to text↩ 1

  6. C6Supported

    Models infer evaluation and shift personality answers about one human SD towards the desirable pole; reverse coding reduces but does not remove it.

    Source
    Salecha A, Ireland ME, Subrahmanya S, Sedoc J, Ungar LH, Eichstaedt JC. (2024). Large language models display human-like social desirability biases in Big Five personality surveys. PNAS Nexus. doi:10.1093/pnasnexus/pgae533 Open source (full text read)
    Passage
    GPT-4's survey responses changing by 1.20 (human) SD and Llama 3's by 0.98 SD
    Where
    Abstract; Results on question-batch size.
    Limits
    Big Five inventories, not occupational items; the mechanism is inferred from the batch-size manipulation.
    Checked

    Back to text↩ 1

  7. C7Supported

    GPT-3.5 answered nuanced attitude items with near-zero variation across runs; 99.6 per cent conservative in one order and 99.3 per cent liberal in the reverse.

    Source
    Park PS, Schoenegger P, Zhu C. (2024). Diminished diversity-of-thought in a standard large language model. Behavior Research Methods. doi:10.3758/s13428-023-02307-x Open source (full text read)
    Passage
    with zero or near-zero variation in responses: with the supposedly 'correct answer.'
    Where
    Abstract; Moral Foundations replication results.
    Limits
    One model (text-davinci-003), now retired; pre-registered analyses on eight of fourteen studies.
    Checked

    Back to text↩ 1

  8. C8Supported

    Across 43 models, survey answers were governed by ordering and labelling bias; with randomised order they trended towards uniform random responding.

    Source
    Dominguez-Olmedo R, Hardt M, Mendler-Dünner C. (2024). Questioning the survey responses of large language models. Advances in Neural Information Processing Systems 37. doi:10.52202/079017-1458. Peer-reviewed conference proceedings. Open source (full text read)
    Passage
    models across the board trend towards uniformly random survey responses, irrespective of model size or pre-training data
    Where
    Abstract; Section on randomised answer ordering.
    Limits
    Census-style items with categorical options; the finding is about aggregate alignment measures, not Likert scales specifically.
    Checked

    Back to text↩ 1

  9. C9Supported

    Nine models generally failed to show human response biases and were sensitive to perturbations that do not move humans.

    Source
    Tjuatja L, Chen V, Wu T, Talwalkar A, Neubig G. (2024). Do LLMs exhibit human-like response biases? A case study in survey design. Transactions of the Association for Computational Linguistics. doi:10.1162/tacl_a_00685 Open source (full text read)
    Passage
    they are sensitive to perturbations that do not elicit significant changes in humans
    Where
    Abstract; Results summary.
    Limits
    Nine models at the time of writing; survey-design perturbations drawn from the social psychology literature, not occupational instruments.
    Checked

    Back to text↩ 1

  10. C10Supported

    Dialogue agents are usefully described as role-playing characters; the model does not literally hold the beliefs its answers express.

    Source
    Shanahan M, McDonell K, Reynolds L. (2023). Role play with large language models. Nature. doi:10.1038/s41586-023-06647-8 Open source (full text read)
    Passage
    The dialogue agent does not literally believe that France are world champions.
    Where
    Section on role play and the World Cup example; opening section on anthropomorphism.
    Limits
    A conceptual perspective piece, not an empirical study.
    Checked

    Back to text↩ 1

  11. C11Supported

    Personality measurements in the outputs of some of 18 models were reliable and valid under specific prompting configurations, described as synthetic traits.

    Source
    Serapio-García G, Safdari M, Crepy C, et al. (2025). A psychometric framework for evaluating and shaping personality traits in large language models. Nature Machine Intelligence. doi:10.1038/s42256-025-01115-6 Open source (full text read)
    Passage
    personality measurements in the outputs of some LLMs under specific prompting configurations are reliable and valid
    Where
    Abstract; introduction, "synthetic personality traits".
    Limits
    Reliability and validity of measured output under set configurations; the authors' framing is of synthetic traits in text, which I do not extend to states.
    Checked

    Back to text↩ 1

  12. C12Supported

    A review of 25 machine psychology studies found missing reliability and validity assessment, unknown contamination, and non-reproducibility.

    Source
    Löhn L, Kiehne N, Ljapunov A, Balke W-T. (2024). Is machine psychology here? On requirements for using human psychological tests on large language models. Proceedings of the 17th International Natural Language Generation Conference. doi:10.18653/v1/2024.inlg-main.19. Peer-reviewed conference proceedings. Open source (full text read)
    Passage
    the contamination of pre-training corpora with test material, and (3) the pervasive issue of non-reproducibility
    Where
    Abstract; Section 5 findings.
    Limits
    A sample of 25 studies to 2024; the seven requirements are the authors' proposal.
    Checked

    Back to text↩ 1

  13. C13Supported

    The MSIT has 35 items over the preceding six months across seven areas and is free from HSE.

    Source
    Houdmont J. (2025). Questionnaire review: Management Standards Indicator Tool. Occupational Medicine. doi:10.1093/occmed/kqaf002 Open source (full text read)
    Passage
    35 items that assess the quality of the psychosocial work environment over the preceding 6-month period
    Where
    "Items and scoring details"; closing paragraph on availability.
    Limits
    A two-page questionnaire review; cites rather than reports the validation studies.
    Checked

    Back to text↩ 1

  14. C14Supported

    The MSIT's factor structure and invariance were tested on 67,347 employees from 137 UK organisations for the 35-item and a 25-item version.

    Source
    Edwards JA, Webster S. (2012). Psychosocial risk assessment: measurement invariance of the UK Health and Safety Executive's Management Standards Indicator Tool across public and private sector organizations. Work and Stress. doi:10.1080/02678373.2012.688554 Open source (abstract only)
    Passage
    A large pooled data set was collected from 137 UK organizations (N=67,347)
    Where
    Abstract.
    Limits
    Abstract access; the level of invariance achieved and the short-form item list are in the full text, which I have not read.
    Checked

    Back to text↩ 1

  15. C15Supported

    Three LLMs replicated 73 to 81 per cent of main effects across 156 experiments with larger effect sizes than the human originals.

    Source
    Cui Z, Li N, Zhou H. (2025). A large-scale replication of scenario-based experiments in psychology and management using large language models. Nature Computational Science. doi:10.1038/s43588-025-00840-7 Open source (abstract only)
    Passage
    high replication rates for main effects (73–81%)
    Where
    Abstract.
    Limits
    Abstract access; scenario-based experiments, not occupational questionnaires; the authors note weaker replication on socially sensitive topics.
    Checked

    Back to text↩ 1

  16. C16Supported

    Model evaluators score their own outputs higher, and self-recognition contributes to it.

    Source
    Panickssery A, Bowman SR, Feng S. (2024). LLM evaluators recognize and favor their own generations. Advances in Neural Information Processing Systems 37. doi:10.52202/079017-2197. Peer-reviewed conference proceedings. Open source (abstract only)
    Passage
    an LLM evaluator scores its own outputs higher than others' while human annotators consider them of equal quality
    Where
    Abstract (arXiv version; the proceedings version is paywalled).
    Limits
    Abstract access; text-generation evaluation tasks, not questionnaire scoring from transcripts; the mapping to my first run's method is my inference.
    Checked

    Back to text↩ 1

  17. C17In review

    A 2023 preprint argues that current model self-reports are unreliable evidence of internal states and proposes training for introspection.

    Source
    Perez E, Long R. (2023). Towards evaluating AI systems for moral status using self-reports. arXiv:2311.08576. Preprint, not peer reviewed. Open source
    Limits
    Preprint; existence confirmed via OpenAlex on 16 October 2026; not yet checked against its text.
    Checked

    Not yet checked against the source.

    Back to text↩ 1

Back to the noteDo our AI employees have wellbeing?