Hacker News: curio

New comment by curio in "Facebook employee responds on robots.txt controversy"

curio — Thu, 17 Jun 2010 20:05:14 +0000

thanks for taking the time to respond to this. i definitely understand where you're coming from and am glad to hear the robots.txt is being updated.

New comment by curio in "Facebook employee responds on robots.txt controversy"

curio — Thu, 17 Jun 2010 18:53:03 +0000

but their robots.txt says that you're welcome to scrape it:

  User-agent: *
  Disallow: /ac.php
  Disallow: /ae.php
  Disallow: /album.php
  Disallow: /ap.php
  Disallow: /feeds/
  Disallow: /o.php
  Disallow: /p.php
  Disallow: /photo_comments.php
  Disallow: /photo_search.php
  Disallow: /photos.php

New comment by curio in "Facebook employee responds on robots.txt controversy"

curio — Thu, 17 Jun 2010 18:50:07 +0000

incredibly easy.

and not only can the bot lie, it can disregard the robots.txt file altogether. just like the terms of service document for humans, you can choose to disregard it & deal w/ the consequences (blocked IP's, lawsuit, etc).

robots.txt is just a version of the TOS that computers can read.

New comment by curio in "Facebook employee responds on robots.txt controversy"

curio — Thu, 17 Jun 2010 18:38:28 +0000

yes, using the user-agent:

http://en.wikipedia.org/wiki/Robots_exclusion_standard

New comment by curio in "Facebook employee responds on robots.txt controversy"

curio — Thu, 17 Jun 2010 18:03:39 +0000

this is a really important issue.

this new model that facebook is trying to push isn't scalable. it favors the big guys and it's bad for the open web.

you shouldn't have a TOS that contradicts your robots.txt. period.

New comment by curio in "Facebook changes the rules for the public web"

curio — Wed, 16 Jun 2010 07:46:44 +0000

probably not, but that's irrelevant. the point is facebook's robots.txt says you can crawl it but their TOS says you can't. facebook have already shown they are willing to sue anyone who tries to crawl without permission. this new model that facebook is trying to use isn't scalable, it favors the big guys and is bad for the open web.

New comment by curio in "Simple Way to Extract GET Params from a JavaScript Script Tag"

curio — Sat, 12 Jun 2010 17:30:39 +0000

you can also do this with a regular expression. here's the blog post i wrote on it a year ago. i use one function to grab get variables from either window.location or from a script tag. 5 lines of code.

http://www.onlineaspect.com/2009/06/10/reading-get-variables...

New comment by curio in "Feel free to type anywhere"

curio — Fri, 04 Jun 2010 19:40:12 +0000

and just like chatroulette or most anonymous forums, the tone quickly deteriorates to the level of graffiti in a public bathroom stall.

New comment by curio in "Student fined for bringing university into disrepute with FitFinder site"

curio — Tue, 01 Jun 2010 05:06:24 +0000

he should drop out of school and use it as an excuse to get on every tv station and written about in every paper in the country.

who wants a degree from such a dumb school anyway. walk away. some of the best companies of our time have been built by drop-outs.

New comment by curio in "Perfect email regex finally found"

curio — Fri, 28 May 2010 23:38:53 +0000

practically speaking, most respectable email services will have an MX record. i tested it on about 30,000 emails & a simple MX lookup caught about 40 gmial's and only 1 false positive.

New comment by curio in "Panera lets customers pay what they want"

curio — Wed, 19 May 2010 04:46:16 +0000

that reminds me of the girl who takes advantage of the zappos free returns policy and gets 5 new pairs of shoes every singles week only to mail them back a few days later. but zappos doesn't cut here off -- she is important evidence that they are serious about their policy & for zappos there are enough honest people out there to more than make up for her.

New comment by curio in "LifeLock CEO’s Identity Stolen 13 Times"

curio — Wed, 19 May 2010 03:42:08 +0000

and while we're on the topic of how broken SSN's are...

why is it that the most important piece of identification you receive in your life, that you also need to keep forever, is printed on a crappy piece of paper???

EventVue post-mortem

curio — Fri, 05 Feb 2010 22:28:09 +0000

Article URL: http://blog.eventvue.com/post/372936164/post-mortem

Comments URL: https://news.ycombinator.com/item?id=1105091

Points: 47

# Comments: 6

New comment by curio in "PubSubHubBub Security Concerns"

curio — Sun, 22 Nov 2009 06:24:38 +0000

It sucks that comments aren't enabled on that post. You can see the responses from the community (including Brett Slatkin one of the authors) on the Google Group for PuSH:

http://groups.google.com/group/pubsubhubbub/browse_thread/th...

New comment by curio in "IPhone Developers: We use our iPhones in bed, allow us to disable landscape"

curio — Thu, 24 Sep 2009 23:02:57 +0000

YES!

New comment by curio in "RSSCloud Vs. PubSubHubbub: Why The Fat Pings Win"

curio — Thu, 10 Sep 2009 13:29:07 +0000

That's an important point. You can see how PuSH handles it here: http://pubsubhubbub.googlecode.com/svn/trunk/pubsubhubbub-co...

New comment by curio in "RSSCloud Vs. PubSubHubbub: Why The Fat Pings Win"

curio — Thu, 10 Sep 2009 06:56:38 +0000

Crap - don't know how that happened. I've reposted the article with the correct link if you care to thumb that one up instead.

RSSCloud Vs. PubSubHubbub: Why The Fat Pings Win

curio — Thu, 10 Sep 2009 06:27:10 +0000

Article URL: http://www.techcrunch.com/2009/09/09/rsscloud-vs-pubsubhubbub-why-the-fat-pings-win/

Comments URL: https://news.ycombinator.com/item?id=814774

Points: 3

# Comments: 1

RSSCloud Vs. PubSubHubbub: Why The Fat Pings Win

curio — Thu, 10 Sep 2009 05:41:11 +0000

Article URL: http://www.techcrunch.com/2009/09/09/rsscloud-vs-pubsubhubbub-why-the-fat-pings-win/

Hacker News: curio

New comment by curio in "Facebook employee responds on robots.txt controversy"

New comment by curio in "Facebook employee responds on robots.txt controversy"

New comment by curio in "Facebook employee responds on robots.txt controversy"

New comment by curio in "Facebook employee responds on robots.txt controversy"

New comment by curio in "Facebook employee responds on robots.txt controversy"

New comment by curio in "Facebook changes the rules for the public web"

New comment by curio in "Simple Way to Extract GET Params from a JavaScript Script Tag"

New comment by curio in "Feel free to type anywhere"

New comment by curio in "Student fined for bringing university into disrepute with FitFinder site"

New comment by curio in "Perfect email regex finally found"

New comment by curio in "Panera lets customers pay what they want"

New comment by curio in "LifeLock CEO’s Identity Stolen 13 Times"

EventVue post-mortem

New comment by curio in "PubSubHubBub Security Concerns"

New comment by curio in "IPhone Developers: We use our iPhones in bed, allow us to disable landscape"

New comment by curio in "RSSCloud Vs. PubSubHubbub: Why The Fat Pings Win"

New comment by curio in "RSSCloud Vs. PubSubHubbub: Why The Fat Pings Win"

RSSCloud Vs. PubSubHubbub: Why The Fat Pings Win

RSSCloud Vs. PubSubHubbub: Why The Fat Pings Win

New comment by curio in "Coding for the unexpected"