A 4
   B 5
   B 8
   C 9
   A 6

Regarding the 1gram file at https://storage.googleapis.com/books/ngrams/books/googlebook...

This is the result I got

   3| 1742563279

   q)\ts d:(!/)(" II";"\t")0:`:1gram
   q)\ts 1#desc sum each group d
   1897 134218176
   371 238872864

   k)\ts d:(!/)(" II";"\t")0:`:1gram
   k)\ts desc:{$[99h=@x;(!x)[i]!r i:>r:. x;0h>@x;'`rank;x@>x]}
   k)\ts 1#desc (sum'=:d)
   1897 134218176
   0 3152
   372 238872864

What is the smart way to do this in kdb+?

This is my naive, sloppy 15min approach.

Warning: Noob. May offend experienced k programmers.

   k)`t insert+:`k`v!("CI";"\t")0:`:tsvfile
   k)f:{select (*:k),(sum v) from t where k=x}
   k)a:f["A"]
   k)b:f["B"]
   k)c:f["C"]
   k)select k from a,b,c where v=(max v)

Unless I am mistaken this parasitic cruft only serves Google, not end users.

Below is quick and dirty program to filter out the above. Replace .com with .cctld as needed.

Requirements: cc, lex

Usage:

   curl -o 1.htm https://www.google.com/search?q=xyz
   yyg < 1.htm > 2.htm
   your-ad-supported-web-browser 2.htm

   flex -Crfa -8 -i g.l;
   cc -Wall -pipe lex.yy.c -static -o yyg;

   %%
   [^\12\40-\176]
   \/url[?]q= 
   "http://www.google.com/gwt\/x?hl=en&u=" 
   "&"[^\"]* 
   %%
   main(){yylex();}
   yywrap(){}

amphtml does look great in a text-only browser that does not load iframes automatically.

I did something similar with a youtube downloader I wrote, using a long chain of seds. It is not as beautiful as Python but it is smaller and faster.

When I shared it with HN the youtube-dl author called it "unmaintainable". By who? I have had no problems maintaining it. :)

Requirements: lex, cc

Usage:

   fetch -4o yc.htm https://news.ycombinator.com
   yc < yc.htm 

    flex -Crfa -8 -i yc.l;
    cc -Wall -pipe lex.yy.c -static -o yc;

    #define jmp BEGIN
    #define p printf
    #define x yytext
   %s aa bb cc dd ee ff gg hh
   %s ii jj kk ll mm nn oo 
   aa "span class=\"rank\""
   bb "a href=\""
   cc score_........
   dd \>
    /* #include  */
    /* #include  */
   %%
   [^\12\40-\176]
   , p("%2c");
    /* rank (dont care) */
   {aa} jmp aa;
    /* [1-9][^<\.]* p("\n%s,",x);jmp bb; */
   [1-9][^<\.]* p("\n");jmp bb; 
    /* url */
   {bb} jmp cc;
   http[^"]* p("%s,",x);jmp dd; 
    /* title */
   
{dd} jmp ee;
    /* [^><]* p("%s,",x);jmp ff; */
   [^><]* p("%s",x);jmp ff;
    /* host (omit) */
    /* points (dont care) */
   {cc} jmp gg;
   {dd} jmp hh;
    /* [1-9][^<> p]* p("%s,",x);jmp ii; */
   [1-9][^<> p]* p(",");jmp ii; 
    /* user */
   {bb} jmp jj;
   http[^"]* p("%s,",x);jmp kk;
    /* time (dont care) */
   {bb} jmp ll;
    /* http[^"]* ; */
   http[^"]* jmp mm;
    /* unix time (dont care) */
    /* [1-9][^<]* { 
    time_t t0; time_t t1; time_t t2;
    t1=time(&t0);
    t2=parsedate(x,&t1,0);
    p("%d,",t2); 
    jmp mm;
    } */
    /* item */
   {bb} jmp nn;
    /* http[^"]* p("%s,",x);jmp oo; */
   http[^"]* p("%s",x);jmp oo;
    /* comments (dont care) */
   {dd} jmp oo;
    /* [1-9d][^ <]* p("%s",x);jmp 0; */
   [1-9d][^ <]* jmp 0;
   .
   \n
   %%
   main(){ yylex();}
   yywrap(){ p("\n");}

What if compiler writers will not fix the problem?

Write own register allocator: http://cr.yp.to/qhasm.html

Bonus: allow use of a "portable assembly language" to generate assembly, being respectful of the fact that users might not all be using computers with the same CPU architecture.

This could be like assembler with C-like operators and structure.

In addtion to facilitating portability it might also make writing in assembler a little easier.

Reminds me of Bell Labs' LIL: http://www.ultimate.com/phil/lil/tut.html

Anyway, how is the difference significant?

A localhost cache can point at a custom root.zone. The user can make her own authoritative nameserver assignments for any given zone or domain. Zone files can contain wildcards.

Responses can also be rewritten on the fly.

The end user can exercise full control over what is and is not a "valid" domain name. She can prevent her applications from ever receiving an "NXDOMAIN" response.

Maybe I am missing something but this "test" seems brittle; it only tests ICANN DNS.

Example below filters out all URLs for a specific section of the paper.

   test $# = 1 ||exec echo usage: $0 section

   curl -o 1.json https://static01.nyt.com/services/json/sectionfronts/$1/index.jsonp
   exec sed '/\"guid\" :/!d;s/\",//;s/.*\"//' 1.json

Personally, I think a protocol like netstrings/bencode is better than JSON because it better respects the memory resources of the user's computer.

Every proposed protocol will have tradeoffs.

To me, RAM is sacred. I can "parse" netstrings in one pass but I have been unable to do this with a state machine for JSON. I have to arbitrarily limit the number of states or risk a crash. As easy as it is to exhaust a user's available RAM with Javascript so too can this be done with JSON. Indeed they go well together.

If you are suggesting that developers, regardless whether they develop mobile apps or ransomware, will start relying less on DNS, I respectfully disagree.

Someone else in this thread commented how reliance on DNS makes systems "fragile". With that I strongly agree.

The same old assumptions will continue to be made, such as the one that DNS, specifcally, ICANN DNS, is always going to be used.

How to break unwanted software? Do not follow the assumptions.

For example, to break a very large quantity of shellcode change the name or location of the shell to something other than "/bin/sh".[1]

Will shellcoders switch to a "robust statistical model" instead of hard coding "/bin/sh"?

Someone once said that programmers are lazy. Was he joking?

1. Yes, I know it may also break wanted third party software. When I first edited init.c, renamed and moved sh I was seeking to learn about dependencies. I expected things to break. That was the point: an experiment. I wanted to see what would break and what would not.

   * routine stolen from gimpel
   * algorithm from peck and schrack 

    define('p(s)t,n,c,k','p_init') :(p_end)
   p_init n = size(s)
    r = array('2:' n, 0)
    &alphabet len(n) . y
    x = array('2:' n, y)
    k = n + 1
   p_0 k = k - 1
    x[k] len(1) . s1 tab(k) . s2 = s2 s1  :s(p_0) 
    define('p(s)i,k') 
    p = s :(return)
   p k = size(s)
   p_1 s = replace(x[k],y,s) :f(p_2)
    r[k] = r[k] + 1
    k = eq(remdr(r[k], k),0) k - 1 :s(p_1)
    p = s :(return)
   p_2 define('p(s)t,n,s1,s2','p_init') :(freturn)
   p_end

   * example: all permutations of string abcdefgh
    s = 'abcdefgh'
   abc output = p(s) :s(abc)f(end)
   end

    define('r(s,ors)c,f,s1,a,d,os') 
    :(r_end)
   r ors rtab(1) len(1) . c :f(freturn)
    s (span(c) | null) . f =
    s arb . s1 len(1) . d c = :f(r_1)
    r = s1 f c d s :(return)
   r_1 ors break(c) . os
    r = r(s,os) f :s(return)f(freturn)
   r_end

    s = 'abcdefgh' 
    output = s
   x01 output = r(output,s) :s(x01)
   end

I see this usage often where it seems like

  grep pattern1 | grep -v pattern2 

  sed -n '/pattern1/d;/pattern2/p'

  sed '/pattern1/!d' | sed '/pattern2/d'

  sed -n 's/pattern1pattern2//g;/pattern2/p'

For example look at the "grep -v" usage here:

https://github.com/thomwiggers/qhasm/raw/master/qhasm-arm

Is there something wrong with using

   sed '/^op:livefloat80:/d'

  sed 's/\$/#/g'

  tr '$' '#'

  old_pids=$(ps -A -opid,args | grep haproxy | egrep -v -e 'grep|reload-haproxy' | awk '{print $1}' | tr '\n' ' ')

  old_pids=$(exec ps -A -opid,args |sed -n '/sed/d;/reload-haproxy/d;/haproxy/{s/ .*//;p};'|tr '\n' ' ')

To avoid a crash, users can mount potentially malicious filesystems in userspace, i.e. users can run kernel drivers like ffs outside of the kernel. This feature comes from a non-Linux kernel. I have read this may be able to work on Linux too but I have never tried it.

Here is a fizzbuzz in Spitbol. Probably not the best way to do it.

      var a = 0;
      start
      +a = a + 1;
      break
      +lt(a,101) :f(end);
      x01
      +x = (remdr(a,3) + remdr(a,5)); eq(x,0) :s(x06);
      x02
      +y = remdr(a,3); eq(y,0) :s(x04);
      x03
      +z = remdr(a,5); eq(z,0) :s(x05)f(x07);
      x04
      +output = 'fizz' :(start);
      x05
      +output = 'buzz' :(start);
      x06
      +output = 'fizzbuzz' :(start);
      x07
      +output = a :(start);
      end
      

He also has pages on APL and k.

So many of these instructions I never see anyone use, ever.

No doubt there are some folks using them, but as mere mortals, do the rest of us really need all these features to control our small amount of commodity hardware? As a user, I have modest goals. Is it not true that Torvalds wrote his kernel with a similarly modest goal in mind: control over his own commodity computer?

The situation resembles that of an overcomplex software program where a majority of the features are unused by an even larger majority of its users. In other words the depth of features benefit only the very few people who use them.

Given the choice between several alternatives with differing levels of features I tend to opt for software that is less featureful and hence more simple. Call me simple-minded if you wish. The same goes for processors, although when it comes to hardware how much choice to we really have as end users? (Hobbyist boards excluded.)

For a taste of some non-x86 assmbler, I enjoyed experimenting with a MIPS simulator still found at spimsimulator.sourceforge.net. I can report that the non-GUI portion at least still compiles relatively cleanly on BSD. This simulator has been mentioned on HN several times.

I have a no problem with using a processor with fewer instructions even if I have to sacrafice something by making that choice -- I leave it to the experts to detail those sacrafices and why I would be a fool to make them. NB: I am already a fool so it may not be worth the effort.

How many HN readers have tried RISC-V? A poll for those who have not: will RISC-V inspire you to purchase a new computer?

The strange "counterargument" I commonly see on HN to any suggestion that Microsoft closed source software could potentially be unsafe for use on an internet-connected computer is that the company has "improved" since some earlier 1990's/2000's time period.

Are these commenters suggesting that other, open source operating system choices have not also improved since that time period? Should one consider how much did each respective system need to improve?

(By "other, open source operating system choices", I mean the ones that were able to connect to the internet for years before Gates decided the www was something his company should be interested in and to copy the TCP/IP stack from an open source kernel into the Windows kernel).

Are there convincing arguments why Microsoft deserves special treatment compared to the open source alternatives, i.e., why their users should not be permitted to freely evaluate the Windows kernel or Office source code via the public web? Are there compelling reasons why MS users should not be allowed i.e. given the option to edit/remove source code they are uncomfortable with and recompile? Consider the effects of limiting the number of people who can find and fix defects in a product.

Does closed source status of Windows make Microsoft's software superior to the longstanding open source operating system alternatives?

Works for me without Javascript.

Any projects that attempted to do this sort of overlay for UUCP?

Incidentally, I like the way the author presents this project. Usage examples, thoughtful use cases, comparisons, protocol description and packet formats.

No hype.