OpenAI’s child exploitation reports increased sharply this year
- Thread starter JournalBot
- Start date
My goodness I hope OpenAI isn't using its own tech for this.
This morning I searched for a quote and Google's Gemini insisted it was from Die Hard. It wasn't, the AI was just confused, but it provided "helpful" analysis as to why the character said the quote which he didn't say. I kept looking because I am familiar with Die Hard and did not believe its lies, eventually finding the true source of the quote. Someone who was less familiar with the movie probably would have found Gemini's inaccurate summary plausible, since it made up new details to support its argument.
That's a minor goof by a different company, but it's a prime example of why this technology should not be used lightly by any company for something as important as moderation and reporting.
Upvote
80
(81
/
-1)
It's good they're reporting it but something tells me they're still ingesting that material for their next model's training data.
Upvote
19
(23
/
-4)
How much of this is real? How much of it is not. I have heard reports of people sending images of a mole to their kids pediatrician, then triggering a response. Sometimes an AI can think that an adult is a kid or, a kid throwing a baseball is illicit.
Upvote
29
(29
/
0)
At a minimum the data needs to be used for Reinforcement learning Human Feedback (RLHF). This image was a violation, that image was not... Use that data to make the filters more accurate.
Upvote
-12
(0
/
-12)
Somewhat off topic, but that shit drives me nuts. People trust the output of these LLMs way too blindly.
Recently at work for compliance purposes I had to confirm whether a piece of test & measurement equipment had nonvolatile storage, and if so could being securely wiped or reset. Since this is a niche product from a small vendor, I couldn’t find any details and had to reach out to them.
While I was working on this back and forth via email over a couple weeks my coworker also offered to do some research on it. He came back to me a day later confident that the device didn’t have nonvolatile storage, only a ROM for the firmware and RAM for operation. When I asked where he got the detailed explanation he had sent me, he said ChatGPT. Ugh.
I am not making an official compliance decision based on ChatGPT output, especially since I was pretty confident it was wrong. (Spoiler alert: it was.)
Upvote
78
(78
/
0)
I wonder how many of these reports are of users uploading images of (clothed) people, and trying to use it like a nudifier app.
Upvote
15
(15
/
0)
If they can report CSAM they can report significant threats of self-harm and threats of violence just like police, doctors, schoolteachers and other mandated reporters.
It’s not privacy they are protecting, it’s profits.
It’s not privacy they are protecting, it’s profits.
Upvote
36
(41
/
-5)
Seems like plenty of room for malicious compliance if OpenAI was in the mind to do that.
I was having it ingest and draw pictures based on my writing, and I’d periodically get a terms violation that I didn’t understand, that may be explained by it finding scenes where the characters are in the shower or bath.
One more thing to worry about.
I was having it ingest and draw pictures based on my writing, and I’d periodically get a terms violation that I didn’t understand, that may be explained by it finding scenes where the characters are in the shower or bath.
One more thing to worry about.
Upvote
10
(10
/
0)
When you're not an expert in a topic, your brain automatically falls back on general heuristics of what expert answers are generally shaped like and sound like when deciding what outside information to trust. And hey, look at that, generally being shaped like and sounding like whatever you ask is exactly what LLMs are good at!Somewhat off topic, but that shit drives me nuts. People trust the output of these LLMs way too blindly.
Recently at work for compliance purposes I had to confirm whether a piece of test & measurement equipment had nonvolatile storage, and if so could being securely wiped or reset. Since this is a niche product from a small vendor, I couldn’t find any details and had to reach out to them.
While I was working on this back and forth via email over a couple weeks my coworker also offered to do some research on it. He came back to me a day later confident that the device didn’t have nonvolatile storage, only a ROM for the firmware and RAM for operation. When I asked where he got the detailed explanation he had sent me, he said ChatGPT. Ugh.
I am not making an official compliance decision based on ChatGPT output, especially since I was pretty confident it was wrong. (Spoiler alert: it was.)
Upvote
45
(45
/
0)
Yikes that’s a scary thought. Could they really be that stupid?
Upvote
3
(3
/
0)
Having been on the vendor side of documenting storage in weird devices for secure erasure, yeah, no, you need reality (and proprietary internal design files) to get that result right!While I was working on this back and forth via email over a couple weeks my coworker also offered to do some research on it. He came back to me a day later confident that the device didn’t have nonvolatile storage, only a ROM for the firmware and RAM for operation. When I asked where he got the detailed explanation he had sent me, he said ChatGPT. Ugh.
I am not making an official compliance decision based on ChatGPT output, especially since I was pretty confident it was wrong. (Spoiler alert: it was.)
As for OpenAI, they got themselves this popular, now they really need to be dealing with the reality of the problematic user content that they've always had in house from pulling things from the internet just now that it's coming in the front door.
Upvote
13
(13
/
0)
Agreed. I do this too - my "best guess" based on my 50+ years of living. I'm right more often than not, but I've hard to start saying "I THINK this is probably true" for culture / music trivia because apparently my voice sounds too confident to the rest of my family or something. Or they're used to me being 99% right on technical stuff. And I am wrong way more than I'd like on the non-technical stuff. For technical stuff, I'm clear in my head on whether I know something or not. But - for example, I misattributed some 80s new wave bands as british that were actually from the US, and my wife did the opposite (thinking they were from the US when they were british), that sort of thing.
But with "the internet", people are expecting it to be 100% correct when it answer with confidence. I've seen a similar problem with Gemini when asking about "curses, foiled again" - something the antagonist (Snidely Whiplash) of the show "Dudley Do-right" (part of Rocky and Bullwinkle) said when Dudley foiled his plans. A co-worker claims Gemini told him that it was said by Dudley Do-right, and despite it having been like 30+ years since I had last seen it, I knew that wasn't right and double and triple checked. And when asked just SLIGHTLY differently, Gemini came up with Snidely Whiplash.
For a "make me an itinerary for visiting Hollywood for a movie buff", it's probably good / great 98% of the time. For actual retrieval of facts that aren't found 1 billion times on the internet - don't trust these things as far as you can throw them.
Upvote
20
(20
/
0)
From what I remember, the original LAION set used by Stability for Stable Diffusion 1.x contained CSAM in some small amount, which was cleaned up pretty quick after that. When you're busy hoovering up all the information, you probably aren't paying too close an attention to what.
Upvote
13
(13
/
0)
I'm left wondering what percentage of CSAM related requests were made by the AIs involved. Since those sorts of things are likely in their training data, I see no reason why they wouldn't occassionally spit that sort of thing out.
Last edited:
Upvote
3
(4
/
-1)
Upvote
9
(9
/
0)
Sora gets upset if you do certain things, if the video has a kid and you try to modify for example to drop the kid, it won’t let you and give you some kind message about children blah blah. I think even try to pie in the face the kid and it won’t let you. Now, that doesn’t mean that gpt won’t.
Upvote
0
(0
/
0)
For anyone who may be interested, the quote was from a two second clip of the TV show Community where a character mixes up the famous "Jesus wept" and "Alexander wept..." lines to comedic effect. It was used in some YouTube video I was watching and I thought it was funny, so I wanted to see where it was from.
In Die Hard, Hans Gruber ironically misquotes Plutarch's reference to Alexander and it's played straight, but for some reason Gemini decided that Gruber actually made the Jesus/Alexander joke I was asking about, and since it doesn't understand the joke, it speculated that Gruber was motivated to say this by his burgeoning megalomania.
Delicious word salad, completely devoid of nutrients.
Upvote
13
(13
/
0)
While it's probably a good thing that CSAM content is being flagged and reported on the whole (if they're not false positives that is), this is also your regularly scheduled reminder that what you type into these services is in no way whatsoever private and you must assume it WILL be used by the AI companies. Either blindly for training purposes, or targeted if need be.
Now I wonder what happens if a FISA court order gets served to OpenAI...
Now I wonder what happens if a FISA court order gets served to OpenAI...
Upvote
8
(8
/
0)
Upvote
4
(4
/
0)
Somewhat off topic, but that shit drives me nuts. People trust the output of these LLMs way too blindly.
Recently at work for compliance purposes I had to confirm whether a piece of test & measurement equipment had nonvolatile storage, and if so could being securely wiped or reset. Since this is a niche product from a small vendor, I couldn’t find any details and had to reach out to them.
While I was working on this back and forth via email over a couple weeks my coworker also offered to do some research on it. He came back to me a day later confident that the device didn’t have nonvolatile storage, only a ROM for the firmware and RAM for operation. When I asked where he got the detailed explanation he had sent me, he said ChatGPT. Ugh.
I am not making an official compliance decision based on ChatGPT output, especially since I was pretty confident it was wrong. (Spoiler alert: it was.)
The other guy is WAY more efficient, though. If I were you, I would watch out...
Upvote
5
(5
/
0)
LLMs are "passably good" at many things. But they are mindblowingly competent at bamboozling people with a knowledge deficit in particular subjects. And we ALL belong to that class in nearly all subjects. Truly A match made in heaven that will keep delivering for a loooong time ro come...
Upvote
7
(7
/
0)
The simplest explanation is probably that this stuff was deliberately mislabeled to hide it online. And assuming that this happens (extra controls), would make the mass scraping pretty much dead in the water from the get go. This would also mean that, apart from being extremely diluted, the captioning mismatch would reduce its ability to contribute to the creating of this kind of stuff. It's just not a serious concern, only an easy entry point to attacking the activity of scraping. A subject that I'm not addressing here in any way, btw.
Upvote
4
(4
/
0)
It's an important reminder about privacy or lack thereof. Our personal information is being used against us now and almost certainly will be much more in the future unless we find a way to do something about it.
Upvote
7
(7
/
0)
They are so consistently wrong, I am beginning to think that there's a Dunning Kreuger virus out there that tends to infect clusters of GPUs running LLMs.
Upvote
1
(1
/
0)
There is a video making the rounds of a cop arresting a guy for trespassing based on a facial match software and the man was totally a false positive who had an ID and paystub and the cop still arrested the guy just because he blindly believed the software.Somewhat off topic, but that shit drives me nuts. People trust the output of these LLMs way too blindly.
Recently at work for compliance purposes I had to confirm whether a piece of test & measurement equipment had nonvolatile storage, and if so could being securely wiped or reset. Since this is a niche product from a small vendor, I couldn’t find any details and had to reach out to them.
While I was working on this back and forth via email over a couple weeks my coworker also offered to do some research on it. He came back to me a day later confident that the device didn’t have nonvolatile storage, only a ROM for the firmware and RAM for operation. When I asked where he got the detailed explanation he had sent me, he said ChatGPT. Ugh.
I am not making an official compliance decision based on ChatGPT output, especially since I was pretty confident it was wrong. (Spoiler alert: it was.)
Upvote
5
(5
/
0)
Isn't that just humanity in general but specifically starting with the LLM/gen-AI developers working at OpenAI, Anthropic, Google, Microsoft et. al. ?
The virus writes itself and it's the LLM.
Upvote
3
(3
/
0)
Well, what would the alternative be? Can you imagine how many tens of thousands of full time workers would be needed to monitor every interaction? And do you suppose a battalion of minimum wage Pakistanis would do a better job anyway?
Upvote
1
(1
/
0)
