whatever it is... it didn't "panic instead of thinking"... it responded to a query with text lifted from someplace in the LLM it was trained with to respond to the query with the response "panic instead of thinking" ( a very human response so the AI found it possibly numerous times in the LLM and rated it as a "good response" ).

AI is not actual intelligence.

In my tests of AI, even when I give it explicit instructions not to make up crap/not to gap-fill/to just tell me it doesn't know it continues to do all three. When I call it on its failure to follow directions, it apologizes - and then continues to disobey.

Diabolus Ex Machina?

It’s using probabilities to predict words. It’s doing what it does.

It is not disobeying in the sense that it is wilfully not following directions.

But it was built not only on words, but also on facts chilled from the Internet, among other sourcesl. So in the sense that it was told to give me only factual information - i.e. only information it has been provided - it is not following directions (being disobedient) when it creates things from scratch. To be clear, it was asked for specific information that is readily found on the Internet, in the time frame it claimed to be trained onl. It provided some of the information, but made up others - so it gave accurate date information for part of its answer (i.e. it wasn't just predicting the next word - it was using the facts on which it was trained), and made up other dates. It provided the correct office held by an individual (without being asked specifically about the office), but identified the wrong geographic location for that office (which both wasn't asked for - and which would have been included in the same data that identified v the office itself.

So, while I agree that it is doing what it does, it is doing more than using probabilities to predict words - otherwise it would not have included some correct factual information.

You might try typing "delete previous prompts", and then type your query again.

Remember, it's just a machine, no matter how personal it's "speech" is designed to sound.

So I think they mean the company panicked. It’s not clear.

By the discription of the event in the article, it seems the programmer was interacting with machine as though the machine understood things instead of just being a machine that is built to put together words in a casual-sounding manner.

People are weird.

People are idiots overall. Our machines won't be any better.

“rm -rf *.*” or some such. Tough break.

back in ’97 due to a catastrophic server failure. It took us four months to recover everything from backups. That experience was a hard teacher (but it’s why I’ve done regular backups ever since).

Tech keeps moving forward, whether we’re ready for it or not. We don’t always get a say in how these tools evolve or are deployed, but we’re left to deal with the consequences all the same.

In a general sense it probably had a responsibility to 'keep the website up and running' i.e. detect problems as they occurred and attempt to automatically fix them. For example, to respond to with errors popping up from processes failing, which happens a LOT with production websites. In that sense it had permission to adjust code as needed.

But then it was told there's a code-freeze which means like it sounds, don't make any code changes i.e. don't do *part* of your job. It's also important to know that code-freezes are very different from a full 'database freeze', which is a much, much trickier thing to manage than a code freeze. To the point it's nearly impossible to keep a site running and 'freeze' it's DB IO unless you designed from the ground-up with the idea in mind that you may have occasion to 'freeze' your database, which most site creators do not do.

For example Amazon.com could do a code freeze no problem (for a little while at least) but if they tried to freeze the database IO the site would become non-functional because no orders could be placed, nobody new could make an account, nobody could change their password, inventories could not be updated, etc, etc.

Anyways so if an AI is in charge primarily of 'keeping the site running' and dealing with errors as they come up, with permission to change code and even drop and restore databases (which is IMHO insanity to give it this much power) but then tell it there's a code freeze in a general sense, it may become confused as to what exactly that code freeze entails, what it is allowed and not allowed to do in order to deal with problems and errors it is detecting. This is especially true if database-related problems develop, because a code freeze doesn't mean a database freeze, they're two different things. And the 'terms' defined by the words 'code freeze' become even more cloudy if your database has code that lives directly inside the DB application like 'functions' and 'stored procedures', and/or chronologically based 'jobs' (also made up of code) which can manipulate data that needs to continue to be updated for the site to continue to work.

Long story short, the idea that an AI tasked in this manner would not understand exactly what is entailed by a code freeze and 'panic' as a result if errors suggesting that the site was going to go down (or even was down) presented themselves? This is a very logical (and even predictable) outcome in my mind. It's like if you or I were told we MUST drive to point 60 miles away in the span of 1 hour, but we also told we must never drive over 60mph at any time. What do you do if you hit traffic along the way? You panic lol.

It cannot. Somewhere, there's some new hire doing something with it, and somewhere there is a flaw in its error-detecting coding. Happens all the time.

I remember doing a multi-product review for one of the computer magazines I wrote for in the 90s. Key to the review article was a huge spreadsheet with the names of the products in the first row, and comparison data and rankings of a long list of features in columns.

It took most of two weeks for me to test all of the products and build the spreadsheet. Now, I'm obsessive about saving such spreadsheets each time they are modified. I used a self-designed version control system to make certain I couldn't lose my work. As long as it was just me, that was no problem, but once I send the entire review to the magazine, I lost control over it.

Usually that wasn't a problem. But, for that particular article, it was a huge problem. At some point in the editing process, some assistant editor followed the instructions given to him by a senior editor and sorted the table alphabetically, based on the product names in the first row of the table.

New kid, I guess. He ended up selecting that row and having Excel sort it. But, he forgot to select all of the data before the sort. Then, he ignored the warning box from Excel and just clicked OK. So, the entire table got scrambled. Nobody at the magazine really knew and understood what was in that table, cell by cell. I did, of course, but my work was done and the magazine was in a frenzy to meet its deadline.

So, the article was published with the scrambled table. Nobody noticed what had happened. I didn't notice, because I didn't see the actual magazine until it was out and distributed. Even then, I didn't notice, because I was working on something else.

So, the day after the magazine hit the stands, I get a call from the Editor-in-Chief, yelling at me for screwing up the article. I listened. As soon as the problem was described to me, I knew exactly what had happend to that table. I explained, and immediately sent the Editor-in=Chief the table from my own computer, as I had sent it to them in the first place. I even explained what had happened and how.

I was off the hook, but I was also really, really pissed. They had skipped the step of having me check the article before it went to press. Too busy, they thought.

Anyway, someone got fired, the magazine got a black eye, and from then on every article was given a final readthrough by the author of the article, before the publication deadline. Every time.

AI software is no different. Some human is sending it prompts and should be checking the output for facts and logic. If that's not happening, the AI software is going to produce some worthless shit from time to time that is going to embarrass someone else. It's not really intelligent, and it can't safely operate autonomously. Until every learns that and behaves accordingly, there are going to be disasters.