I could be wrong, I am not a Prime Video user, but the result I got was that there are 217 movies in Prime Video from Ebert's great movies.

   links -dump 1.html|grep -c Prime.Video 

This solution uses a generous 87s delay to retrieve the Amazon pages. There are 328 films listed as "great movies" on rogerebert.com. As such, the script, named "1.sh", needs 8h to complete, e.g., the time while you are at work or sleeping. No cookies, no state, no problems.

   Usage: sh -c 1.sh > 1.html

   #!/bin/sh

   curl -HUser-Agent: -H'Accept: application/json' --compressed 'https://www.rogerebert.com/great-movies/page/[1-16]?utf8=%E2%9C%93&filters%5Btitle%5D=&sort%5Border%5D=newest&filters%5Byears%5D%5B%5D=1914&filters%5Byears%5D%5B%5D=2020&filters%5Bstar_rating%5D%5B%5D=0.0&filters%5Bstar_rating%5D%5B%5D=4.0&filters%5Bno_stars%5D=1'|grep -o "/reviews/great-movie-[^\\]*"|sed 's/.reviews.great-movie-//'|sort|uniq|while read x;do y=$(echo $x|sed 's/-/+/g');echo $x;curl -s --compressed -HUser-Agent: https://www.amazon.com/s/?k=$y 2>/dev/null|grep -m1 -C4 a-link-normal.a-text-bold;sleep 87;done|sed '/^[^< ]/s/.*/@&,/;1s|.*|&|;s/ *//;/^$/d;/^[@<]/!s|$||;1s/@//;s/@/
/'

Absolute favourite episode hands down, The Cage. According to Shatner's autobiography, NBC called the pilot "too cerebral" and "too intellectual".

There has always been a place for commerce and marketing on the web."

Not really true as I remember it. The web opened up to the public in 1993. There was no commerce and marketing in the beginning. Even by 1996 while commerce and marketing may have existed, e.g., Amazon founded in 1995, its place was in the background. As I rememember the early web, the foreground, the "starting point" or "portal", was something like Yahoo! You had to pick a topic (direction) that you wanted to go in. For example, if you were after music, you might end up browsing the Internet Underground Music Archive. The "front page" of the portal was predominantly non-commercial, mostly generic headings for topics. If you wanted to search out something commercial, no doubt you could but the initial starting point was intellectual curiosity. This is IMO what has been lost over time with regard to web use: intellectual curiosity and the ability to actually satisfy it. (A fun tangent here is the collections of inane queries that people type into Google. These are simultaneously hilarious and disturbing.)

As an experiment have a look at the Yahoo! page today. It is full of low quality mainstream "news". There is zero attention to intellectual curiosity. Nothing to see here, folks, but here is the latest news. For part 2 of the experiment, run a Google search for the term "music". The results are dominated by YouTube. Every result is directly or indirectly commercial (either selling something or conducting surveillance and serving ads), except one: Wikipedia. The chances of someone new to the web not following a link to YouTube or some other Google-controlled domain would seem almost nil.

The "onboarding" process for new web users is very different today than it was in the early 1990's. Perhaps it is still possible to approach the web with a sense of awe and wonder, pondering "What is out there?" However a new web user is scant likely to end up on a non-commercial website besides Wikipedia. What is out there? Surveillance, ads and an endless supply of soon-to-be-obsolete Javascript du jour.

"... and critically: there's no one to hold responsible for getting it wrong."

Could this be part of "AI"'s appeal? A dream of absolving businesses and individuals from accountability.[2]

1. "What's more, artificial research teams lack an awareness of the specific business processes and tasks that could be automated in the first place. Researchers would need to develop an intuition of the business processes involved. We haven't seen this happen in too many areas."

2. Including the ones who designed the "AI" system.

This sentence exemplifies the perspective to which I referred.

   #/bin/sh
   test -s max-PMID||echo 32449615 > max-PMID;read x < max-PMID;h=pubmed.ncbi.nlm.nih.gov;
   test ${#x} -eq 8||rm max-PMID;sed -i "s/[0-9]\{8\}/$x/" $0;
   case $1 in update) mkfifo 1.fifo 2>/dev/null;test -p 1.fifo||exec echo need 1.fifo;
   (grep "https://raw.githubusercontent.com/lemire/fastbase64/master/s...

]]>Mon, 25 May 2020 21:01:35 +0000https://news.ycombinator.com/item?id=23304948x3blahhttps://news.ycombinator.com/item?id=23304948https://news.ycombinator.com/item?id=23304948"It's not super productive to try and understand this stuff through analogy to computers and algorithms."
You mean like this: https://ds9a.nl/amazing-dna
Having worked in both industries I prefer working with wet science people. For some reason they generally have a much healthier perspective on life.  Their work is humbling because it is, and will forever be, full of unsolved mysteries, not simply because it is challenging.  The other folks, whether they call themselves "scientists" or "engineers" or "developers" or "coders" or whatever, are working with something that as far as I can see has no inherent connection to the natural world, other than being a production of the human mind.  Perhap that affects the perspective many of them have on life.  For example, how common among them is this belief that all things, not simply computers, can be thoroughly understood and mastered. Note this is pure opinion, not fact, and I am generalising; there are exceptions to every generalisation.
]]>Mon, 25 May 2020 06:53:10 +0000https://news.ycombinator.com/item?id=23298676x3blahhttps://news.ycombinator.com/item?id=23298676https://news.ycombinator.com/item?id=23298676What if PubMed had something like Google's "I'm feeling lucky"?
What if we could explore PubMed by selecting a random PubMed URL instead of searching?
This script generates a random PubMed URL.  
To do this we need to know the maximum PMID number in the PubMed database.  The current max is included in the script and will be saved in a 9-byte file named "max-PMID" when the script is run.
If run with the argument "update" it will search for a newer max PMID.
If a newer max PMID is found, the script updates the number in the max-PMID file and in the script itself.
An alternative is to use the ftp server[1] to find the max PMID; I noticed the latest ftp update was missing new PMID's caught by this script.
If run without any arguments it selects a random PMID between 1 and the max and outputs a URL.
uses socat, GNU sed and requires a fifo named "1.fifo" 
1. ftp://ftp.ncbi.nlm.nih.gov/pubmed/updatefiles/
       #!/bin/sh
       test -s max-PMID||echo 32446294 > max-PMID;read x < max-PMID;x=$((x-1));h=pubmed.ncbi.nlm.nih.gov;
       test ${#x} -eq 8||exec echo weird max-PMID;sed -i "/test/s/echo [0-9]\{8\} /echo $x /" $0;
       case $1 in update) mkfifo 1.fifo 2>/dev/null;test -p 1.fifo||exec echo need 1.fifo;
       (grep "For those not using Javascript (SSRN uses third party scripts), link to PDF: https://poseidon01.ssrn.com/delivery.php?ID=5550640311191050...

]]>Fri, 22 May 2020 20:48:55 +0000https://news.ycombinator.com/item?id=23276810x3blahhttps://news.ycombinator.com/item?id=23276810https://news.ycombinator.com/item?id=23276810The long line of sed is out-of-date and thus "broken".  For something simpler that works, try this:
   nyt tr |sed 's/ *//;/ travel.html

This will produce a simple web page of titles and URLs for each article page.
An interesting point of discussion might be the amount of third party cruft on the template article page versus the more dynamic front page.  When Javascript is disabled, on each article page all images display and there are no ads.  Downloading any video in the page is as simple as
   curl -O `grep -o https://[^\"]*mp4 article.html`

]]>Wed, 20 May 2020 02:44:49 +0000https://news.ycombinator.com/item?id=23242786x3blahhttps://news.ycombinator.com/item?id=23242786https://news.ycombinator.com/item?id=23242786"It's text, just send the damn text."
They only send what the user requests.
Using a software program that makes automatic requests that you are not easily in control of, e.g., a popular web browser, might give the impression that they control what is sent.
They do not control what is sent.  The user does.^1
The user makes a request and they send a response.
One of the requests a fully-automatic web browser makes to NYT is to static01.nyt.com
Personally, as a user who prefers text-only, this is the only request I need to make.  As such I don't really need a heavily marketed, fully-automatic, graphical, ad-blocking web browser to make a single request for some text.^2
    #! /bin/sh

    case $1 in
    world        |w*)  x=world       # shortcut: w
    ;;us         |u*)  x=us          # shortcut: u
    ;;politics   |p*)  x=politics    # shortcut: p
    ;;nyregion   |n*)  x=nyregion    # shortcut: n
    ;;business   |bu*) x=business    # shortcut: bu
    ;;opinion    |o*)  x=opinion     # shortcut: o
    ;;technology |te*) x=technology  # shortcut: te
    ;;science    |sc*) x=science     # shortcut: sc
    ;;health     |h*)  x=health      # shortcut: h
    ;;sports     |sp*) x=sports      # shortcut: sp
    ;;arts       |a*)  x=arts        # shortcut: a
    ;;books      |bo*) x=books       # shortcut: bo
    ;;style      |st*) x=style       # shortcut: st
    ;;food       |f*)  x=food        # shortcut: f
    ;;travel     |tr*) x=travel      # shortcut: tr
    ;;magazine   |m*)  x=magazine    # shortcut: m
    ;;t-magazine |t-*) x=t-magazine  # shortcut: t-
    ;;realestate |r*)  x=realestate  # shortcut: r
    ;;*)
    echo usage: $0 section
    exec sed -n '/x=/!d;s/.*x=//;/sed/!p' $0
    esac

    curl -s https://static01.nyt.com/services/json/sectionfronts/$x/index.jsonp

   Example: Make simple page of titles, article urls and captions, where above script is named "nyt".

    nyt tr |  sed '/\"headline\": \"/{s//
/;s/\".*/<\/p>/;p};/\"full\": \"/{s//
/;s/..$/<\/p>/;p};/\"link\": \"/{s///;s/ *//;s/\".*//;s|.*|&|;p}' > travel.html

    firefox ./travel.html

Source: https://news.ycombinator.com/item?id=22125882
The truth is that they are just sending the damn text.  However you are voluntarily choosing to use a software program that is automatically making requests for things other than the text of the article, i.e., "cruft".
1. The Google-sponsored HTTP/[23] protocol is seeking to change this dynamic, so if websites sending stuff to you without you requesting it first bothers you, you might want to think about how online advertisers and the companies that enable them might use these new protocols.
2. However I might use one for for viewing images, watching video, reading PDFs, etc., offline.  Web browsers are useful programs for consuming media.  It is in the simple task of making HTTP requests that their utility has diminished over time.  The user is not really in control.
]]>Tue, 19 May 2020 21:56:45 +0000https://news.ycombinator.com/item?id=23240777x3blahhttps://news.ycombinator.com/item?id=23240777https://news.ycombinator.com/item?id=23240777"Which brings us to the question - what is the point of all this? These platforms are all losing money. Just think of all the meetings and lines of code and phone calls to make all of these nefarious things happen which just continue to bleed money. Why go through all this trouble?
How did we get to a place where billions of dollars are exchanged in millions of business transactions but there are no winners? My co-host Can and my restaurant friend both defaulted to the notion "delivery is a shitty margin business" when discussing this post.
You have insanely large pools of capital creating an incredibly inefficient money-losing business model.
It's used to subsidize an untenable customer expectation.
Third-party delivery platforms, as they've been built, just seem like the wrong model, but instead of testing, failing, and evolving, they've been subsidized into market dominance.
The more I learn about food delivery platforms, as they exist today, I wonder if we've managed to watch an entire industry evolve artificially and incorrectly."
A contrary view, from 4 days ago, arguing third party food delivery market is not created by VC, the startups are not over-funded and that they are delivering splendid returns to investors.
https://news.ycombinator.com/item?id=23171915
]]>Mon, 18 May 2020 04:22:37 +0000https://news.ycombinator.com/item?id=23218305x3blahhttps://news.ycombinator.com/item?id=23218305https://news.ycombinator.com/item?id=23218305There is a link to the PDF on the page.  No ACM membership required.
https://dl.acm.org/doi/pdf/10.1145/3317550.3321445
]]>Sat, 16 May 2020 10:27:58 +0000https://news.ycombinator.com/item?id=23202257x3blahhttps://news.ycombinator.com/item?id=23202257https://news.ycombinator.com/item?id=23202257"Also known as rump kernels a name inspired by the infamous purge of royalists from Parliament following the English Civil War this process involves creating a fork of an existing kernel codebase and manually purging it of the components deemed unnecessary to the target unikernel."
That appears to come from the FAQ on Github for "rumprun" dated 2015.
Rump kernels were introduced in NetBSD in 2009.  It was an described then as an acronym for runnable userspace meta programs.
https://blog.netbsd.org/tnf/entry/runnable_userspace_meta_pr...
I have not used rumprun for Linux but I still use the rump utilities included with NetBSD.
]]>Sat, 16 May 2020 09:03:57 +0000https://news.ycombinator.com/item?id=23201752x3blahhttps://news.ycombinator.com/item?id=23201752https://news.ycombinator.com/item?id=23201752"They are redirecting those customers to their platform where they can purchase from a competitor or they can charge us a fee to provide food to that same customer.  Whereas the customer could have just gone to our website and ordered directly from our online ordering store and we would not be charged a commission to GrubHub or DoorDash."
Here he refers to Grubhub and DoorDash as "competitors" with the restaurant.
Yet at the conclusion of the story he refers to them as "partners" with the restaurants who pay them.
"DoorDash and GrubHub have a long history of screwing their partners."
These companies are "middlemen".  That description in fact fits a large number of "tech" companies.  It is a common theme when we look at the "tech" industry.
A few days ago food delivery, including Grubhub, was discussed on HN.
https://news.ycombinator.com/item?id=23169597
One commenter argued that companies like Grubhub exist in "fake markets" that would not exist were it not for VC funding.
Another argued that these markets are not fake because these companies have managed to be profitable and deliver returns to investors.
Not sure what you all have experienced, in my experience with restaurants and delivery, the owners always seem to prefer customers who order directly from the restaurant, not via a middleman.  However, I can sense that they feel compelled to sign up with the middlemen for fear of losing business.
HN commenters seem to enjoy comparing first page search results on Google versus alternatives such as DDG.  Try this one out.  Search for your favourite restaurant.  On Google, I see more middlemen than I see on DDG, including a result for "[restaurant-name].business.site" which I believe belongs to Google.  I don't see that result on DDG.
Imagine if the telephone book Yellow Pages listed multiple numbers for each business, some of them not belonging to the business but to middlemen who would "make calling easier".  Even more, imagine it changed the order of the listings according to a "secret" methodology instead of presenting them in alphabetical order.
]]>Sat, 16 May 2020 07:45:54 +0000https://news.ycombinator.com/item?id=23201296x3blahhttps://news.ycombinator.com/item?id=23201296https://news.ycombinator.com/item?id=23201296One can use "port knocking" for more than remotely opening ports.  It is a crude form of messaging in its own right, that can be based on some pre-determined code, like Morse code.  The "secret knock" need not open any ports.  It can be simply a message to the person (or program) reading the logs, to be translated according to the pre-determined code.  Actions might be taken in response to the message, or not.
It is possible to do similar things with haproxy.  Configured to listen for TLS connections on a large number of ports, look for a secret combination of custom headers and values, then, if found and matching, forward to a localhost ucspi tcpserver on the backend.  The tcpserver may then execute some program, for example sshd or pfctl.
http://cr.yp.to/ucspi-tcp.html
]]>Fri, 15 May 2020 02:34:05 +0000https://news.ycombinator.com/item?id=23188237x3blahhttps://news.ycombinator.com/item?id=23188237https://news.ycombinator.com/item?id=23188237"How do you reconcile this narrative with the fact that Grubhub turned a net profit every available year until 2019 per their SEC filings."
Easy.  Take away the VC funding and see if anyone wants to try or can manage to start a similar company.
In most cases, these companies are just middlemen.  Many folks do not see middlemen as "legitimate" businesses, even though the middlemen make money.
Hosts and parasites can each thrive.  However only one can survive on its own.  Ideally hosts would prefer to live free of parasites.
It should be expected that some consumers or producers will want to cut out the middlemen.  We cannot reasonably expect everyone to appreciate those middlemen for the success they may have in taking a cut (and collecting data on consumer behaviour).
]]>Thu, 14 May 2020 04:54:36 +0000https://news.ycombinator.com/item?id=23175470x3blahhttps://news.ycombinator.com/item?id=23175470https://news.ycombinator.com/item?id=23175470