cross-posted from: https://programming.dev/post/51288619

Hi All,

Due to some ongoing issues with harassment campaigns, we’ve had to setup a rudimentary monitoring system for all new users.

  • When a user’s signup is accepted, they will be automatically enrolled into the monitoring system. The admins team may also add accounts manually if they have been given a strike.
  • The system will monitor all posts, comments and DMs sent by new users, and bring them to the attention of the admin team if it appears suspicious. In egregious cases, it will auto-remove posts and comments if required, but a human admin will always review and reverse any false positives as soon as required.
  • Once we have validated that the user is not a spammer, they will be removed from the system.

We don’t want to go into too much detail on how it all works, but we can say that all the processing is being done locally on the instance server. For most of you, this wont have any impact, but some of you have been impacted by the systems false positives. It is also a good time to point out that DM messages are not private, and should not be used for anything that requires strong privacy.

There will likely be teething problems, but we are actively working on improving the bot to minimize impact and we are always open to feedback.

  • Grimy@lemmy.world
    11·
    6 days ago

    I welcome all ideas to curb abuse but reading DMs seems a bit much. I figure users can report abusive DMs, which could then be viewed instead of looking at them before hand?

    That being said, I find it hard to think of new users sending legitimate DMs. I have done it once I think during my entire time on Lemmy.

    • UlrikHD@programming.dev
      3·
      6 days ago

      I’ll copy the answer I provided in the main thread over on !meta@programming.dev, but to also add, the tool does not monitor all users. It only monitors newly created user accounts on programming.dev for x amount of time so that we know that the account isn’t being made to abuse or spam. We do not want to have this tool monitoring regular users.

      Copied reply about why DM monitoring is needed:

      From the PR standpoint, I don’t think it’s worth it, and it’d be better to just leave it on reports.

      The problem is that we can’t rely on reports of DMs since lemmy doesn’t federate them to us.

      E.g.
      -> troll@programming.dev makes a new account and sends harassment to victim@lemmy.instance
      -> victim@lemmy.instance reports the DMs from troll@programming.dev
      -> We, the admins of programming.dev, do not see this report because lemmy does not federate the DM report and troll@programming.dev can continue harassing others because we never find out about it.

      This isn’t just a theoretical, it happened just last month that one of our users (1 day old account) sent rape and death threats (which were reported), and we found out about it by pure chance when talking to admins from the other instance.

      And just to clarify, the tool only automatically monitors new accounts, i.e. accounts that are being registered today. If you account is more than a few weeks old, the tool doesn’t monitor any of your activity.

      • Grimy@lemmy.world
        1·
        6 days ago

        Ohhh, I see the problem with the non-federating reports. That does make these measures much more understandable, although it sounds like it’s not receiving the reports that needs fixing overall.

        • UlrikHD@programming.dev
          3·
          6 days ago

          It’s an open issue on github from 2024, it doesn’t seem to be a priority. This tool allows us to react faster than reports though, and hopefully remove some problematic content before it’s seen by others.