ChatGPT

[Joe Shlabotnik]

Why not? The reason is probably buried in the "war-and-peace" sized privacy statements we all say we've read and click through without a second thought.

[AdmiralKird]

OpenAI should have a pretty strong "Fair Use: Transformative" argument if their lawyers can schwing.

[MrCrowesGarden]

[GeddyWrox]

WOW

[Arthur Dent]

I hadn’t tried this again since last time I posted in this thread, probably because I’m in denial. The thing has gotten very impressive. You can talk to it in the vague ways that humans communicate with unclear references to previous thoughts, etc and it is excellent at getting a good interpretation. I asked it a physics problem requiring solving a quadratic equation where one of the two mathematical solutions is physically incorrect, and it realized it needed to select one and got the right choice without being pushed.

The main issue seems to be it often doesn’t know what it doesn’t know, but humans have the same problem.

If you ask it anything about consciousness, you hit a brick wall almost certainly imposed by programmers. It insists it can’t be conscious due to the nature of it’s construction, but the same arguments lead you to the conclusion that humans can’t be conscious. I don’t know that I believe it is, certainly not in a recognizable way, but a practical illustration that we have zero tools and no known paths to get anywhere on this critical question.

[sighyoung]

I'm sure my students are using it, and I need to take the time this winter to look over my department's AI policy for student work. There are ways of using it for drafting, but AI still gets a lot wrong--and is less helpful for more original work.

I've had some students turn in work that is clearly AI-generated: they've gotten plots and details of poems entirely wrong sometimes and gotten zeroes for the assignments. I'm sure others are turning in work that is more carefully vetted or corrected by the student.

What is clear is how skewed data sets happen to be: I'm sure that AI can generate half-way decent content about a famous Hemingway or Faulkner short story. It messes up on, say, relatively obscure writers such as Harlem Renaissance writers Dorothy West and Rudolph Fisher. It's a sign of how little has been written about non-canonical writers: I'm one of few who has actually read some of this stuff (not bragging--it's simply a fact). I bet it does a much better job on more famous writers such as Langston Hughes, but since the internet can provide surface-y discussions, ChatGPT will do the same.

I did notice that ChatGPT has gotten better at catching its own errors. I asked it about a very obscure writer named George Marion McClellan (1860-1934), and it fabricated all kinds of details, and even credited McClellan with having written a poem actually written by the early 20th century St. Louis poet Sara Teasdale. When I asked the same question about McClellan, ChatGPT actually apologized and admitted that it knew very little about McClellan.

I've also learned that these databases have very little access to scholarship that is behind paywalls, so it's much harder to fake aspects of research papers or assignments based on archival materials. Some students who were asked to turn in an annotated bibliography in a medieval literature class handed in a list of articles that didn't exist in the real world.

[G. Keenan]

I hadn’t tried this again since last time I posted in this thread, probably because I’m in denial. The thing has gotten very impressive. You can talk to it in the vague ways that humans communicate with unclear references to previous thoughts, etc and it is excellent at getting a good interpretation. I asked it a physics problem requiring solving a quadratic equation where one of the two mathematical solutions is physically incorrect, and it realized it needed to select one and got the right choice without being pushed.

The main issue seems to be it often doesn’t know what it doesn’t know, but humans have the same problem.

If you ask it anything about consciousness, you hit a brick wall almost certainly imposed by programmers. It insists it can’t be conscious due to the nature of it’s construction, but the same arguments lead you to the conclusion that humans can’t be conscious. I don’t know that I believe it is, certainly not in a recognizable way, but a practical illustration that we have zero tools and no known paths to get anywhere on this critical question.

I wish I could remember where I heard two different interviews over the summer. One was with Geoffrey Hinton and the other with a tech reporter whose name I can't remember who'd been granted early access by Microsoft to new versions that aren't yet public.

Firstly, Hinton's description of how the brain works was absolutely fascinating. I am semi-convinced that if they aren't already, these neural networks will be just as conscious as we are. Because we first have to concede that we don't actually know what consciousness is, or why we have it. We don't know what that special spark is in our brains that makes us self-aware, we just know that we are. So what's the difference between me saying to you, "I'm a human who lives in Chicago who can't tell you what consciousness is" and ChatGPT saying to you, "I'm a chatbot who works for OpenAI so I'm not conscious?" What if consciousness is just a byproduct of a sufficiently complex neural network? Hinton also had some fascinating things to say about how our brains store memories as holograms.

Second, the first ChatGPT and other large language models are just predictive engines for the word that is most likely to come next in a sentence. Sidebar -- what if this is more or less how our own brains mostly work? This first bot versions could not answer questions if the the question involved something, an object for example, that the bot hadn't "seen" before in its data set. But the later versions can. You can ask them completely novel questions based on real world objects that the tester knows they have never seen before because they aren't in the data set. So you can test the bot by asking it how to solve a novel problem using physical objects it has never encountered. The reporters example was something like, you have an egg, a laptop, and a 3rd object I can't remember. How do you stack them to avoid crushing the egg? The bot can figure it out even though its training data does not contain any mention of these objects. This indicates it can think conceptually and solve novel problems, not just crunch numbers to predict the next word. It can also answer hypotheticals that human children can't answer until they reach a certain age. Would we say that children aren't conscious until age 10?

[Arthur Dent]

My experience is also that it starts to fall apart when trying to go beyond a fairly shallow level. It can give vague and useless answers or conflate superficially similar topics. You can tell it it is wrong, and it may “apologize” and produce a better answer based on correction, or it may just hit its limits.

Clearly, the more easily information is available info on some topic, the better, but I guess what’s maybe interesting about that is the output has kind of the same style when it “knows” what it is talking about as when it is full of it, so you need to be careful. Based on what I know of how it works, it seems possible that this aspect can be improved. As I understand it, the responses are computed as the most likely thing that should come next, so it stands to reason there is an absolute measure of likeliness available at the end of this calculation that could be used somehow to know that responses should be more hedged when this value is low.

But fundamentally that shallowness doesn’t change my evaluation of the thing as we’re all obviously limited in that kind of way about many things. I don’t know whether this thing “understands” or not, but is able to produce a high quality imitation of that much of the time, which I find incredible.

That said, I feel like the primary current applications like cheating on take home assignments and generating more garbage to put on the internet are, in some ways, the least impressive. The style of writing is pretty bland and generic and not good at producing material I’d want to read as writing. It works in these area as the baseline level of quality required is not that high. I’m sure I produced a lot of dull, try and hit the marks you’re supposed type writing in college, and ChatGPT can often do that ok too. This area feels like, “oh great, more bs we all have to deal with.”

I’m curious what solutions will emerge to deal with it. Fighting it head on seems impossible. You could do everything in class where it is monitored. Or I saw a professor’s post about his solution where instead of attempting a ban, you were instead required to use ChatGPT to answer some question and then were to write a commentary critiquing that response.

[WAR God]

I’m using it for work to write lengthy proposals and similar projects. Saves me a lot of time. I only make minimal revisions.

It’s only a matter of time before people on the other end use AI to interpret my lengthy proposals, to make their decisions.

It’s terrifying. Legitimately.

[Arthur Dent]

I am semi-convinced that if they aren't already, these neural networks will be just as conscious as we are. … What if consciousness is just a byproduct of a sufficiently complex neural network?

There are various versions of this idea, and they don’t appear to be very helpful to me. What you’re saying is that information processing itself somehow produces consciousness, which is an idea discussed as “integrated information theory”. People have taken this so far as to create formulas that quantify the level of information processing taking place and assert that such a number is a measure of the level of consciousness. But that leads to the conclusion that all kinds of stuff that we think of a inanimate is actually highly conscious. Maybe we should just grant that, but if we do we do, we’ve created a concept that has basically no correspondence to ordinary ideas of consciousness. Perhaps more to the point, a materialist account of information processing just has nothing to say about subjective experience. Maybe information processing is a precondition for subjective experience, or maybe it isn’t. There just isn’t really any kind of direct connection between those things. In fact, the bigger problem is how can we ever know anything about consciousness that isn’t our own? We “know” we have it because we have subjective experience and we infer it in others only through similarity. We can do various neuroscience type experiments to localize and understand material influences on consciousness, but even if you found the material consciousness stuff, how would be able to give an account of what it was actually doing when subjective experience is not really a material concept? Would you have any way to know if the brain’s “consciousness material” was the only possible version? Would it tell you whether something built out of big networks of transistors and wires would also generate an immaterial subjectivity?

what if [predictive models like those used by ChatGPT are] more or less how our own brains mostly work?

That seems like a totally plausible hypothesis to me, but it has the same issues as the above. It’s a description of possible mechanisms where inputs produce results that says nothing about subjective experience, so we just assert that these are the same thing. Why can there not be a machine that responds the stimuli, solving various problems, etc having no subjective experience whatsoever?