Using AI to identify trolls

[John Q. Public]

We've talked about it before and people have accused sites of censorship for moderating or dropping comments sections altogether, but trolling, spamming and off-topic comments are a big, and often expensive, problem for websites.

Well, Google to the rescue. They're testing a system called Perspective that they say will be able to spot problem posts and either flag them or disallow them or let the poster re-think them before posting. It doesn't seem to be quite ready for wide distribution yet - their own example doesn't flag "Your a socialist snowflake!" - but some sites working with them on it are Wikipedia, the New York Times, the Guardian and the Economist.

I'm thinking it might be useful for forums - especially if Google is the one developing it, because Google's opinion of your site is important in terms of being found and in the various definitions of "value" that potential visitors and advertisers might use. Or it could just improve forum conversations by letting a poster know whether his post is likely to be taken well or not. Google's algorithm hates forums, so I'm hoping it might help with that by raising the quality of conversations, if that's possible. Hard to tell on that one since users don't usually have a stake in the site and very often couldn't care less about such things.

Sez here,

Publishers can choose what they want to do with the information they get from Perspective. For example, a publisher could flag comments for its own moderators to review and decide whether to include them in a conversation. Or a publisher could provide tools to help their community understand the impact of what they are writing—by, for example, letting the commenter see the potential toxicity of their comment as they write it. Publishers could even just allow readers to sort comments by toxicity themselves, making it easier to find great discussions hidden under toxic ones.

https://blog.google/topics/machine-lear ... ersations/

Sounds like it could make comments sections more interesting. Or more entertaining, depending on which end you set the slider on.

[Omar Bongo]

I don't know if anyone else here saw it but this week the Internet Movie Database dropped the big one on its discussion forums, poof, everything gone, even the archives. Supposedly because policing the trolls was too much. Maybe this could have prevented that...

[MDDad]

Why do we need special software to identify our trolls? We know who they are.

[John Q. Public]

This is the part that sounds promising to me.

"While we improve the technology, we’re also working to expand it. Our first model is designed to spot toxic language, but over the next year we’re keen to partner and deliver new models that work in languages other than English as well as models that can identify other perspectives, such as when comments are unsubstantial or off-topic."

[John Q. Public]

I don't know if anyone else here saw it but this week the Internet Movie Database dropped the big one on its discussion forums, poof, everything gone, even the archives. Supposedly because policing the trolls was too much. Maybe this could have prevented that...

You're talking about a lot of work on a site the size of IMDB - and a lot of money. Comment moderation has actually become a good sized industry and I hear it pays decently. I don't know how big their forums were but you could be talking about some serious money to stay on top of them - likely more than they brought in.

[Omar Bongo]

The forums were very popular with members, and I'm sure lured many non-members looking for movie reviews and answers to questions to the site as well - people were screaming bloody murder for weeks after IMDB announced the decision to nuke it. Maybe they're saving some money, but they're losing a lot of clicks as well.

[John Q. Public]

It could also be a matter of quality and unpredictability, too. Big buck advertisers are very leery of User Generated Content, regardless of how much traffic or clicks a site gets. Forums and comment sections can be great for traffic but they can be hell for ad rates. I'd imagine a well-directed blog or magazine site without comments would probably pay twice as much as a forum with comparable traffic. The Bots That Be judge pages on their overall quality and they don't separate comments from articles, so the ones without the "Yeah, I seen that to"s and "You're full of it, Omar"s are going to rate much higher than the ones with it.