Rendered at 07:19:45 GMT+0000 (Coordinated Universal Time) with Cloudflare Workers.
socketcluster 10 hours ago [-]
This approach feels wrong. For code, it's obvious now that Claude is adding watermarks through comments because they are way too long. I keep asking Claude to remove and reduce its comments.
Also, I don't think it's useful because AI involvement in the work is a spectrum, not a binary true or false. If I ask Claude to go over my code and fix up typos and clean up and it starts adding comments with hidden watermarks everywhere it makes the tiniest adjustment, it's essentially appropriating my code. Its contribution may be trivial and superficial.
When I use AI on my own projects, the code it generates basically looks like auto-complete to me; exactly what I would have written by hand. So I don't see why it needs to be marked as AI. It's just saving me time. It's very much my own work.
aucisson_masque 9 hours ago [-]
Beside the added comments, why does it matter that everyone know you used ai to write your code ? You’re not ashamed of that, so let everyone know.
unrented7977 6 hours ago [-]
Because they make the code harder to read
LudwigNagasena 10 hours ago [-]
It’s obvious that such boneheaded methods don’t work. So what’s next? First, we need to deal with basic word substitution, which seems theoretically feasible. Then we need to deal with encodings such as Caesar cipher, replacing spaces with zero-width spaces, Base64, etc. Your account will be flagged and reported for outputting obfuscated text. What’s next after that? Ooops, you output too many vim commands instead of outputting text directly, your account is flagged and reported to Europol. You think you can bypass that with Deepseek? No, it will be banned alongside VPN.
Muromec 9 hours ago [-]
Is this before or after we all get confined to 15 minute ghettos and chipped in your timeline? I'm concerned about both and need to prioritize my prepping (not actually prepping, just sitting there being nervious and spreading it around).
mgax 14 hours ago [-]
This is such a waste of time.
If someone wants to bypass this it will be rather simple. Just change the words. If someone wants to avoid fingerprinting they will.
Can’t we just focus on building rather than spending brainpower on these ridiculous sidequests
gonzalohm 13 hours ago [-]
Focus on building what? A future in which only big companies get credit for their work?
bsoqk 13 hours ago [-]
Isn't that what this watermarking allows for? To allow LLM companies to detect that certain text was authored by their tools?
SpicyLemonZest 13 hours ago [-]
There's lots of regulations that are easy to bypass. It's absolutely trivial to, say, pull 30 amps from a home circuit rated for only 15 amps. But most people will just trip the fuse because they don't know what they're doing, and the existence of the regulation makes it easy to prove ill intent for anyone who does know what they're doing but causes harm anyway.
bsoqk 12 hours ago [-]
The way I interpret your comment is that, in the future, it will be enough to prove that someone tried to remove an LLM watermark to put them in jail.
IsTom 12 hours ago [-]
Imagine that somebody sold you text that they mislead you to think is not generated by AI and then trying to convince court that they didn't mislead you intentionally. If they've removed the watermark it stops this kind of defense.
bsoqk 12 hours ago [-]
Why not have our devices generate watermarks for everything? That way, we won’t have any kind of defence.
okanat 11 hours ago [-]
That's what is going on with images taken. iPhone and other cameras now have a per device cryptographic signature. https://c2pa.org/
Soon images taken without this signature will be unpresentable to the court.
braiamp 12 hours ago [-]
That's already on the books... misrepresentation of the product sold... or does the EU doesn't have that?
IsTom 11 hours ago [-]
Generally doing something intentionally or unintentionally might have different penalties.
SpicyLemonZest 12 hours ago [-]
These are corporate regulations, it's not really about putting individuals in jail. I do expect that there could be companies where trying to remove an LLM watermark is a fireable offense, especially in the EU where many kinds of decisions must be made by an accountable human being and may not be delegated to an AI system.
someonebaggy 12 hours ago [-]
If someone causes financial loss by posting LLM text and is then found to have removed a watermark. Everything in civil law is implicitly inside a if(someone harmed && they sue you) {} block.
akersten 13 hours ago [-]
Of course, the implication by comparison to the building code example, that text absent some homeopathic suggestion of provenance causes harm, is absurd at best.
Sent from my iPhone (harm prevention watermark)
protocolture 4 hours ago [-]
>There's lots of regulations that are easy to bypass.
Considering kids already use word substitution and other techniques to avoid plagiarism detectors, is creating a new kind of plagiarism detector weak to word substitution a valid path forward?
Actually, if anything it's just going to compound the false positive rate, making things worse for people who arent cheating.
Analemma_ 13 hours ago [-]
I think some EU regulations have merit and some don't, but going "can't we just focus on building instead of spending brainpower on following the law" is not going to win you much affection. There might be a connection between this pervasive attitude in tech and why now every proposed datacenter construction project is being met with ferocious public opposition.
iamnothere 13 hours ago [-]
This isn’t just a “techbro” attitude. Although they disagree on the specific problem laws in question, basically every political group (at least in the US) takes issue with the law as written. And most agree (72% as polled in 2023) that the law is written to favor the interests of the wealthy. The pushback against data centers isn’t in contrast to this—data centers provide the bulk of their benefits to wealthy investors, and many localities are permitting them against the will of local citizens.
You might also want to consider that many believe the current AI regulation push is a trojan horse for regulatory capture and subsequent domination of the industry by large, well-connected players who are hostile to privacy and individual freedom.
So when you see US people on a US site complaining about a law that affects US industry, especially scrappy startups, consider if maybe there’s more to it.
13 hours ago [-]
Aerroon 10 hours ago [-]
Could this technique be theoretically used to track users themselves? It would be quite ironic if the EU forced tech companies to implement extra tracking, wouldn't it?
Muromec 8 hours ago [-]
You would need to inject too much information and then it's trivially removable by passing it through any noise removal tool of choice. I suspect just deleting the comments and doing the ponytail thingy will be enough.
At some point it would just be verified accounts for social media and some carveouts here and there, and fines for noncomplience in misattribution and mislabeling.
Or we get into zero trust territory where everybody would assume everything is generated and can't be trusted, unless it's their in-group aligned signaling system.
aucisson_masque 9 hours ago [-]
I don’t see how you could implement individual fingerprinting for millions of users, and then make it recognizable. The added processing cost would be insane.
Beside there is nothing in eu réglementation suggesting that.
octoberfranklin 4 hours ago [-]
Yes and very easily too.
I would be astonished if they weren't already doing this.
rapedang3 9 hours ago [-]
[flagged]
k1m 6 hours ago [-]
Worth noting that this is what OpenAI wrote about text watermarking two years ago:
> While it has been highly accurate and even effective against localized tampering, such as paraphrasing, it is less robust against globalized tampering; like using translation systems, rewording with another generative model, or asking the model to insert a special character in between every word and then deleting that character - making it trivial to circumvention by bad actors.
> Another important risk we are weighing is that our research suggests the text watermarking method has the potential to disproportionately impact some groups. For example, it could stigmatize use of AI as a useful writing tool for non-native English speakers.
Mandating and accepting broad use of text watermarking at scale seems prime to enable all sorts of dumb and chaotic downstream effects. Not only do you start to pollute further corpus at scale - hijack text space bandwidth and fidelity - but you open the societal door to gradually add additional pieces of information - such as author identities or ad tracking information - which can then not just directly reveal a single author, but potentially larger graphs of information propagation without it being clear to anyone propagating such a trace.
m-hodges 14 hours ago [-]
> Starting today, API customers globally will be able to opt in to text watermarking for select models. Text watermarking will remain off by default in the API.
> Over the coming weeks, we will add an invisible watermark to eligible ChatGPT and Codex text output in the European Union.
athrowaway3z 13 hours ago [-]
>> Editing can weaken the watermark. In an evaluation of 400-token passages, replacing 10% of words with synonyms reduced detection from about 92% to 66%. Replacing 25% of words reduced it to 17%.
> Claude/codex/deepseek, please replace 25% of words with synonyms or slight rephrasing because i dont like the current version.
Not sure if that counts as: `a solution that's robust against "common alterations and adversarial attacks"`. Is there a sort of adversarial attack that is more common?
yorwba 13 hours ago [-]
If you use a watermarking model to do the synonym replacement, it will rephrase it in a way that is compatible with the watermark...
alienbaby 7 hours ago [-]
I guess they could have it look for any fingerprint first and then preserve it through the new changes
smokel 13 hours ago [-]
Why use watermarking, and not simply add a signature?
Gigachad 10 hours ago [-]
Because the feature is to combat deception. The person generating the text is malicious in this scenario. They will just not include a signature.
It doesn't involve changing the model weights, if that's what you mean.
k__ 9 hours ago [-]
Yes, thank you
GardenLetter27 10 hours ago [-]
I wish we could vote out the EU!
rapedang3 9 hours ago [-]
[flagged]
andriamanitra 9 hours ago [-]
1% false positive rate is completely unacceptable, and if you can bypass it by changing some of the words what's even the point? This is only going to catch low effort slop.
greatgib 13 hours ago [-]
My personal opinion is that they cheated evaluations to be able to release this pretending that it has no meaningful impact.
Otherwise, I don't see any logical explanation that some of their benchmark results would be higher when watermarked. Except if benchmark results are so unstable that they are an useless metric.
yorwba 12 hours ago [-]
As long as variance is nonzero, you won't get the exact same result twice for the same benchmark, so one of the numbers has to be higher and the other lower. If watermarking has no effect on the distribution (by construction, it should have no effect), that's a 50% chance the watermarked model gets the higher number. In this case, it happened 5 out of 8 times, which is hardly unusual.
Y_Y 13 hours ago [-]
I agree that if they're so noisy they need to average over more runs, but maybe they didn't feel the need to bother.
aenis 13 hours ago [-]
Another cookie consent-grade success of the EU.
zetanor 12 hours ago [-]
The near-ubiquitous malicious compliance that took place during the implementation of GDPR (i.e., displaying consent modals instead of just not sending visitors' personal information to 93 third party services) at least had the benefit of bringing to light just how much web service providers view their users as things to be bought and sold. I doubt that text watermarking will result in much of anything at all, besides additional expense.
iririririr 12 hours ago [-]
This is also malicious compliance.
The whole effort started as a way to pin point which work openAI stole from! Now they are redirecting the dicussion to "our sources are magic. let's talk about who is copying from that magic".
All the controversy about this being pointless, is on purpose. They are pushing the pointless thing as a distraction and we keep discussing how pointless it is. no wonder.
Aerroon 10 hours ago [-]
Is https://www.europa.eu also malicious compliance? Because what on earth does the EU commission need a cookie pop up for?
And yet there it is. Maybe the law is flawed?
aucisson_masque 9 hours ago [-]
There are YouTube videos embedded in some eu pages.
aenis 11 hours ago [-]
I keep saying: its worse than that. Companies that actually do not need to ask for consent do - because their compliance departments prefer to play it safe. My current employer genuinely does not share any data with anyone, the only stuff we process is for our own technical purposes, very well within the remit of GDPR. But year after year, a chief data privacy officer after chief data privacy officer, I lose the battle for "lets get rid of the obnoxious cookie consent banner, we dont need it". All big companies are like that.
And of course, cookies are no longer needed for invasive tracking, fingerprinting, and data disaggregation. Been this for years. But hey, bureaucracies never backtrack, so we will have the cookie consent until the thermal death of the universe.
Boltgolt 10 hours ago [-]
> And of course, cookies are no longer needed for invasive tracking, fingerprinting, and data disaggregation. Been this for years. But hey, bureaucracies never backtrack, so we will have the cookie consent until the thermal death of the universe.
The exact method of tracking doesn't matter. Under GDPR you have to ask for consent no matter the technology used. So as long as tracking every single movement remains the norm, we will have consent banners
mjfisher 9 hours ago [-]
Worth knowing cookies are a special case and are covered under the ePrivacy directive, which is distinct from the wider GDPR legislation and specifically deals with storing tracking information in a user's browser.
pembrook 10 hours ago [-]
Good to know, will exclusively move to Chinese models for non-coding tasks.
The idea that producing text with AI needs to be watermarked as if it's a crime by default is backwards nonsense.
To me this would actually be a counter signal.
If you're NOT primarily writing with AI (at least mildly being informed by all of human knowledge distilled), then I will assume your ideas are emotional opinion-based nonsense, like most comments on hackernews, including my own.
richwater 12 hours ago [-]
Just make the models worse for the EU. Don't accept this nonsense that's holdingg back actual work and progress.
Also, I don't think it's useful because AI involvement in the work is a spectrum, not a binary true or false. If I ask Claude to go over my code and fix up typos and clean up and it starts adding comments with hidden watermarks everywhere it makes the tiniest adjustment, it's essentially appropriating my code. Its contribution may be trivial and superficial.
When I use AI on my own projects, the code it generates basically looks like auto-complete to me; exactly what I would have written by hand. So I don't see why it needs to be marked as AI. It's just saving me time. It's very much my own work.
Soon images taken without this signature will be unpresentable to the court.
Sent from my iPhone (harm prevention watermark)
Considering kids already use word substitution and other techniques to avoid plagiarism detectors, is creating a new kind of plagiarism detector weak to word substitution a valid path forward?
Actually, if anything it's just going to compound the false positive rate, making things worse for people who arent cheating.
You might also want to consider that many believe the current AI regulation push is a trojan horse for regulatory capture and subsequent domination of the industry by large, well-connected players who are hostile to privacy and individual freedom.
So when you see US people on a US site complaining about a law that affects US industry, especially scrappy startups, consider if maybe there’s more to it.
At some point it would just be verified accounts for social media and some carveouts here and there, and fines for noncomplience in misattribution and mislabeling.
Or we get into zero trust territory where everybody would assume everything is generated and can't be trusted, unless it's their in-group aligned signaling system.
Beside there is nothing in eu réglementation suggesting that.
I would be astonished if they weren't already doing this.
> While it has been highly accurate and even effective against localized tampering, such as paraphrasing, it is less robust against globalized tampering; like using translation systems, rewording with another generative model, or asking the model to insert a special character in between every word and then deleting that character - making it trivial to circumvention by bad actors.
> Another important risk we are weighing is that our research suggests the text watermarking method has the potential to disproportionately impact some groups. For example, it could stigmatize use of AI as a useful writing tool for non-native English speakers.
https://openai.com/index/understanding-the-source-of-what-we...
> Over the coming weeks, we will add an invisible watermark to eligible ChatGPT and Codex text output in the European Union.
> Claude/codex/deepseek, please replace 25% of words with synonyms or slight rephrasing because i dont like the current version.
Not sure if that counts as: `a solution that's robust against "common alterations and adversarial attacks"`. Is there a sort of adversarial attack that is more common?
Otherwise, I don't see any logical explanation that some of their benchmark results would be higher when watermarked. Except if benchmark results are so unstable that they are an useless metric.
The whole effort started as a way to pin point which work openAI stole from! Now they are redirecting the dicussion to "our sources are magic. let's talk about who is copying from that magic".
All the controversy about this being pointless, is on purpose. They are pushing the pointless thing as a distraction and we keep discussing how pointless it is. no wonder.
And yet there it is. Maybe the law is flawed?
And of course, cookies are no longer needed for invasive tracking, fingerprinting, and data disaggregation. Been this for years. But hey, bureaucracies never backtrack, so we will have the cookie consent until the thermal death of the universe.
The exact method of tracking doesn't matter. Under GDPR you have to ask for consent no matter the technology used. So as long as tracking every single movement remains the norm, we will have consent banners
The idea that producing text with AI needs to be watermarked as if it's a crime by default is backwards nonsense.
To me this would actually be a counter signal.
If you're NOT primarily writing with AI (at least mildly being informed by all of human knowledge distilled), then I will assume your ideas are emotional opinion-based nonsense, like most comments on hackernews, including my own.