Sunday 20 September 2009

Spammers are killing me

I have over the last years developed an experimental socio-semantic bookmarking service as a part hobby and part academic research project. The site can be said to be under early Beta testing. I have a lot of plans for the site but as it is being developed on my spare time the progress is not as fast as I could have hoped for. Also, as I write the project is kind of on the shelf while I explore another exiting hobby project in the area of social location based services.

Here come the spammers
In the middle of July 2009 I suddenly saw a rise in the number of new members signing up on my bookmarking service fuzzzy.com. Surely it's some robots spamming fuzzzy I thought. But after looking into the actions performed I soon figured out the spam where human generated. Based on the type of actions and data/metadata entered I could tell these where not generated solely by automated agents. It very much looked like coordinated spamming from a spam ring. There were no pattern in the IP addresses used. Captchas and human-readable-only questions on the sign up page did not stop them. Looking in the log-files at the seconds between actions, links added in the form of bookmarks and free html text links where added and modified in a typical human workflow. Some spammers also added tags and comments.

Why are they doing this
Obviously Google pagerank is the root of all evil link bombing spam. Often called link bombing, google bombing, spamdexing, referer spam, spammers add links to sites in order to promote a site and make it rank higher on Google and other search engines using page rank and similar ranking algorithms.

Spammers killing me slowly
For the last weeks I have got about 30 spam links every day and the process of removing the spam is killing me. Instead of using my scarce time on development and learning new stuff I am tied down for 10 minutes each day just verifying links and deleting spam. 10 minutes a day is not that much but its the feeling of fighting against a mob of EVIL EVIL EVIL real world spammers that really makes me just feel sad and frustrated. On just about every page on fuzzzy there is now text saying the site is a community site for people interested in web science and web development. Still, people keep bombing the site with spam links.

So what should I do
As the site is still in sort of early closed beta I don't have a bunch of users that can report, moderate and delete spam.
The few options I see are:
  • Close the site until I decide to focus 100% on the site and a real community is built around the site.
  • Keep deleting spam every day.
  • Develop a spam blacklisting service my self.
  • Report the spam to some third party black list.
  • Develop functionality that favours user with high reputation. Links posted by new users are just not shown until the link or the user is voted up or something like that.
If you have any ideas for how to fight the spam please let me know.
The last option or similar approaches seems to be the way to go but it does seem futile to fight the spam mob. If I can free my site of the spammers they will only move on as parasites to new victims. This only shows how primitive the current state of the web really is.

6 comments:

  1. I can sooo understand what you are going through. When I was young (much younger than I am now), I had created one of Austria's first full text search engines (think 1995!).

    After 2 years of happy growing I became a spam target. In the end I could not cope with 200 links per day, and captchas were not even invented then.

    My advice: Cut access completely. Seriously. It is not the 10 minutes per day. It is the daily annoyance.

    ReplyDelete
  2. Hi Robert. Interesting to hear that spamming has been going on since the early days of the web. Regarding your advice, I'm not sure I understand. The spammers are using new accounts each day from new IP-addresses. So I don’t see how I can cut access without also shutting out other new non malicious users.

    ReplyDelete
  3. How about an approach similar to the last item on your list -- and to what some blogging tools use:

    New links have to be approved by you or some type of power-user, unless the poster has a previously approved link (or two or three...)?

    Then create an interface that makes it extremely easy for you to mass approve/delete spam ... and do so no more than once a week.

    If it is really human users creating the links, I'd like to think that that would make them go away. At least after a while. Those f**kers.

    ReplyDelete
  4. Hi Roy,
    Some users are not Spammers even when you think so.
    I'm not a Spammer but you take my honnest use of the utility you offer on fuzzzy.com, for a Spammer activity.
    I was not so happy with you. Now I understand you.
    Regards/FilipeAlvesFerreira#4(1942)

    ReplyDelete
  5. Thank you Trond. I think a variant of the solution your mention is what I will have to go for. At the moment fuzzzy is deliberately shut down just to see if the spammers will move on. If users have earned certain amount of karma points no need for approval should be required.

    ReplyDelete
  6. Hello Filipe, thank you for your understanding. I’m sorry for having deleted your posts.

    This is how I spot spam:
    1.Content added is irrelevant for the majority of the community.
    2.Content is added by a new or a not yet trusted member.
    3.The aim seems to be promoting self interests by linking to sites that he or she has a relation to.
    4.The content of the link destination is not objective or unbiased.
    5.The quality of the content/site is poor.
    6.The content does not seem to be written with an honest and sincere intent to share useful information or knowledge.
    7.The destination site is designed to sell me something.
    8.The user has an excessive use of tags, comments, votes etc on his or her own content.
    9.The user adds several links to the same site or use several URL’s that ultimately lead you to the same site or page.
    As I judged your posts, several of the above criteria’s was fulfilled so I deleted your links with high confidence.

    If the aim of your post is to promote or advertise an idea, methodology, framework, service, product or what ever, that will be beneficial to your self then I consider this, at least to a certain degree to be, spam. If the link posted is not relevant for the community where it is posted then this might not, in a strict sense, be considered spam but none the less irrelevant and should be buried etc.

    If the post added is indirectly beneficial to your self by increasing your status and visibility because you are considered are knowledgeable and sharing person then this is of course ok.

    Please Filipe, I’m keen to know why you so strongly argue that the links you added were not spam.

    ReplyDelete

Spammers will be hunted down and punished.