"The hole they found wasn't hidden. It was sitting there the whole time, waving at me"
In a recent note on parallels between color theory and concepts, there are a couple of ideas about relationships between thoughts (KDOT Post #6).
This article resonates.
- Adjacent thoughts - reinforce and reinforce each other, but may be so similar that distinctions are lost. This is the "Al thinks you are brilliant, you are handsome, strong" and "Can I praise you more, Sir" area. There is value in help finding actual supporting ideas and enrichment, but remain vigilant. Leverage search and expression ability, but do not delegate good judgement.
- Complementary (or contrasting) thoughts - bring clarity in contrast and tradeoff. You may need to pull on the Al to get these as in the article - it depends on the statistics of its internal landscape and resources it pulls on. Al's tend to have favorite topics and positions. Do the effort to explore contrasts and relationships across the information/reality surface. Avoid being only a passenger on any plastic-packaged Al carnival rides. There is a chance the Al may weigh these as opposing its mission to humor you. Variants of "please find the weaknesses and holes in my idea" directives may help.
- Monoconceptual thoughts - build, develop, and retire concepts, but can also destructively interfere and cancel one another. Thoughts may also be lost in larger volumes of thought competing for attention. They may fall outside the Al's response window size in the first or later passes or may be neglected if you don't actively direct. As with contrasting thoughts, Al may not share by default. In thoroughly understanding topics, you will need to actively dig.
Much appreciated, Nick - good food for thought.
ps - We can consider building standard critical prompt arrays. I have one I normally use for self-consistency, hype detection, anti-punchiness, industry contrast, signal-to-noise, and readability rating. The extended idea is to preposition the critical idea lenses to run as a battery much like a unit test array for software. It is not perfect and needs tuning, but it is an attempt and sometimes is helpful.
"Avoid being only a passenger on any plastic-packaged AI carnival rides."
Stealing this. (With attribution. Probably.)
The prompt array as unit test is a strong analogy. My hesitation: unit tests work because you know what correct output looks like. With ideas, the "correct" output is fuzzier. How do you calibrate what passes vs. what fails? Or is it more about flagging things for human review rather than pass/fail?
Genuine question. I'm still figuring out where systematic checks end and intuition has to take over.
I once heard "AI is a brilliant research assistant, with no social skills who doesn't know how to tell you when you are wrong." Always something I try to keep in the back of my mind.
"The prompt array as unit test is a strong analogy. My hesitation: unit tests work because you know what correct output looks like. With ideas, the "correct" output is fuzzier. How do you calibrate what passes vs. what fails? Or is it more about flagging things for human review rather than pass/fail?"
I was thinking less along pass/fail lines than "Have a standard array of shots I can trigger to bang against the content" and see if anything falls out. And there are still concerns the AI will shmooze you.
- Sort of like driving your home-built Baja road racer over your favorite rugged terrain and seeing if anything breaks easily. You might still turn up problems later, but you tried to shake it up.
- In aerospace it would be acoustic and thermal testing of vehicles - not conclusive, but hitting more of the territory.
I suppose the coverage and conclusiveness issue is a general problem in reality. As humans we can't each experience everything personally - there is too much. But we can stack historical observations of others from history, with the caveats of others not seeing everything either and of reporting & correlation problems. And we can try some of the more reachable explorations - like having AI bounce content against its internal topology built from some of that fuzzy asserted experience.
"Readers will obviously understand this" is such a perfect catch. That phrase is always a lie we tell ourselves to avoid doing the work.
(I've definitely written "this is self-explanatory" in drafts. Narrator: It was not self-explanatory.)
The Pre-Mortem is brutal for exactly that reason. Forces you to confront the gap between how it sounds in your head and how it lands in someone else's.
"The hole they found wasn't hidden. It was sitting there the whole time, waving at me"
In a recent note on parallels between color theory and concepts, there are a couple of ideas about relationships between thoughts (KDOT Post #6).
This article resonates.
- Adjacent thoughts - reinforce and reinforce each other, but may be so similar that distinctions are lost. This is the "Al thinks you are brilliant, you are handsome, strong" and "Can I praise you more, Sir" area. There is value in help finding actual supporting ideas and enrichment, but remain vigilant. Leverage search and expression ability, but do not delegate good judgement.
- Complementary (or contrasting) thoughts - bring clarity in contrast and tradeoff. You may need to pull on the Al to get these as in the article - it depends on the statistics of its internal landscape and resources it pulls on. Al's tend to have favorite topics and positions. Do the effort to explore contrasts and relationships across the information/reality surface. Avoid being only a passenger on any plastic-packaged Al carnival rides. There is a chance the Al may weigh these as opposing its mission to humor you. Variants of "please find the weaknesses and holes in my idea" directives may help.
- Monoconceptual thoughts - build, develop, and retire concepts, but can also destructively interfere and cancel one another. Thoughts may also be lost in larger volumes of thought competing for attention. They may fall outside the Al's response window size in the first or later passes or may be neglected if you don't actively direct. As with contrasting thoughts, Al may not share by default. In thoroughly understanding topics, you will need to actively dig.
Much appreciated, Nick - good food for thought.
ps - We can consider building standard critical prompt arrays. I have one I normally use for self-consistency, hype detection, anti-punchiness, industry contrast, signal-to-noise, and readability rating. The extended idea is to preposition the critical idea lenses to run as a battery much like a unit test array for software. It is not perfect and needs tuning, but it is an attempt and sometimes is helpful.
"Avoid being only a passenger on any plastic-packaged AI carnival rides."
Stealing this. (With attribution. Probably.)
The prompt array as unit test is a strong analogy. My hesitation: unit tests work because you know what correct output looks like. With ideas, the "correct" output is fuzzier. How do you calibrate what passes vs. what fails? Or is it more about flagging things for human review rather than pass/fail?
Genuine question. I'm still figuring out where systematic checks end and intuition has to take over.
I once heard "AI is a brilliant research assistant, with no social skills who doesn't know how to tell you when you are wrong." Always something I try to keep in the back of my mind.
Nailed it. Brilliant assistant who'd rather lie to your face than make things awkward.
The whole article is basically: here's how to make it awkward on purpose.
"The prompt array as unit test is a strong analogy. My hesitation: unit tests work because you know what correct output looks like. With ideas, the "correct" output is fuzzier. How do you calibrate what passes vs. what fails? Or is it more about flagging things for human review rather than pass/fail?"
I was thinking less along pass/fail lines than "Have a standard array of shots I can trigger to bang against the content" and see if anything falls out. And there are still concerns the AI will shmooze you.
- Sort of like driving your home-built Baja road racer over your favorite rugged terrain and seeing if anything breaks easily. You might still turn up problems later, but you tried to shake it up.
- In aerospace it would be acoustic and thermal testing of vehicles - not conclusive, but hitting more of the territory.
I suppose the coverage and conclusiveness issue is a general problem in reality. As humans we can't each experience everything personally - there is too much. But we can stack historical observations of others from history, with the caveats of others not seeing everything either and of reporting & correlation problems. And we can try some of the more reachable explorations - like having AI bounce content against its internal topology built from some of that fuzzy asserted experience.
"Readers will obviously understand this" is such a perfect catch. That phrase is always a lie we tell ourselves to avoid doing the work.
(I've definitely written "this is self-explanatory" in drafts. Narrator: It was not self-explanatory.)
The Pre-Mortem is brutal for exactly that reason. Forces you to confront the gap between how it sounds in your head and how it lands in someone else's.
Glad it's working for you.