I thought the Counting Stuff post on synthetic panel data was a good take: "Since user testing is specifically about being surprised by how users react to a given product, using an LLM to simulate a generalized average experience rather misses the point. It's cheaping out on data collection in a way that undermines the purpose of the data collection."
Thanks Fred - I'd not seen that. I like "Once these things stop looking like magic and more like the mishappen blobs of distilled Reddit memes they are, people are going to start asking what can these models do."
(the middle-earth example is brilliant btw)
Thanks, I also polled them on a relaxation of planning laws in the Shire!
I thought the Counting Stuff post on synthetic panel data was a good take: "Since user testing is specifically about being surprised by how users react to a given product, using an LLM to simulate a generalized average experience rather misses the point. It's cheaping out on data collection in a way that undermines the purpose of the data collection."
https://www.counting-stuff.com/the-call-of-llms-is-strong/
Thanks Fred - I'd not seen that. I like "Once these things stop looking like magic and more like the mishappen blobs of distilled Reddit memes they are, people are going to start asking what can these models do."