Hacker News: gtoubassi

New comment by gtoubassi in "Show HN: Qast – Cast anything (files, URLs, screen) to any TV from the CLI"

gtoubassi — Tue, 03 Mar 2026 20:56:53 +0000

This is super cool and totally works on my crappy low end LG tv. WebRTC is awesome. Nice

New comment by gtoubassi in "There's only one Woz, but we can all learn from him"

gtoubassi — Wed, 28 Jan 2026 20:48:17 +0000

Hackers by Steven Levy is an incredible story of the industry’s early years (60-80’s) and the characters that were in it for the “love of the game” vs what is more common now (“status and money”). A lot of heroes like woz, but who are less well known in this day and age (Gosper and Greenblatt!). If you are familiar with and a fan of Dealers of Lightning or Dream Machine, check out Hackers! (this is not a paid endorsement).

New comment by gtoubassi in "Television is 100 years old today"

gtoubassi — Mon, 26 Jan 2026 21:04:27 +0000

"The Last Lone Inventor: A Tale of Genius, Deceit, and the Birth of Television" is a great book detailing the Farnsworth journey.

New comment by gtoubassi in "CubeSats are fascinating learning tools for space"

gtoubassi — Mon, 15 Sep 2025 22:40:58 +0000

If you have any details written up on your kit (in partic what solar you used) I'd appreciate a link. I'm looking to do similar

New comment by gtoubassi in "OpenAI: Streaming is now available in the Assistants API"

gtoubassi — Thu, 14 Mar 2024 03:03:57 +0000

We do the token counting on our end literally just running tiktoken on the content chunks (although I think usually its one token per chunk). Its a bit annoying and I too expected they'd have the usage block but its one line of code if you already have tiktoken available. I've found the accounting on my side lines up well with what we see on our usage dashboard.

New comment by gtoubassi in "LLM Visualization"

gtoubassi — Sun, 03 Dec 2023 21:52:25 +0000

I struggled to get an intuition for this, but on another HN thread earlier this year saw the recommendation for Sebastian Raschka's series. Starting with this video: https://www.youtube.com/watch?v=mDZil99CtSU and maybe the next three or four. It was really helpful to get a sense of the original 2014 concept of attention which is easier to understand but less powerful (https://arxiv.org/abs/1409.0473), and then how it gets powerful with the more modern notion of attention. So if you have a reasonable intuition for "regular" ANNs I think this is a great place to start.

New comment by gtoubassi in "Q-Transformer: Scalable Reinforcement Learning via Autoregressive Q-Functions"

gtoubassi — Wed, 20 Sep 2023 10:59:00 +0000

+1 you beat me to the punch! I think its helpful to start with simple RL and ignore the "deep" part to get the basics. The first several lectures in this series do that well. It helped me build a simple "cat and mouse" RL simulation https://github.com/gtoubassi/SimpleReinforcementLearning and ultimately a reproduction of the DQN atari game playing agent: https://github.com/gtoubassi/dqn-atari.

New comment by gtoubassi in "What the interns have wrought, 2023 edition"

gtoubassi — Tue, 12 Sep 2023 22:19:39 +0000

Token counting is importing when you are injecting fetched data into the prompt to make sure you don't overflow the prompt size (e.g. in retrieval augmented generation). You want to give the LLM as many facts as will fit in the prompt to improve the quality of its response. So even with billions of dollars... token counting is a thing.

New comment by gtoubassi in "A Gambler Who Beat Roulette"

gtoubassi — Fri, 07 Apr 2023 19:11:56 +0000

+1 the book is entertaining (esp for engineers). Also was released under the title "The Newtonian Casino".

New comment by gtoubassi in "Stanford Alpaca web demo suspended “until further notice”"

gtoubassi — Fri, 17 Mar 2023 20:56:39 +0000

https://github.com/antimatter15/alpaca.cpp has links

New comment by gtoubassi in "What is ChatGPT doing and why does it work?"

gtoubassi — Wed, 15 Feb 2023 03:29:47 +0000

I agree this is an incredibly interesting paper. I am not a practitioner but I interpreted the gradient article differently. They didn’t directly find 64 nodes (activations) that represented the board state as I think you imply. They trained “64 independent two-layer MLP classifiers to classify each of the 64 tiles”. I interpret this to mean all activations are fed into a 2 layer MLP with the goal of predicting a single tile (white, black, empty). Then do that 64 times once for each tile (64 separately trained networks).

As much as I want to be enthusiastic about this, it’s not entirely clear to me that it is surprising that such a feat can be achieved. For example it may be possible to train a 2 layer MLP to predict the state of a tile directly from the inputs. It may be that the most influential activations are closer to the inputs then the outputs, implying that Othello-GPT itself doesn’t have a world model, instead showing that you can predict board colors from the transcript. Again, not a practitioner but once you are indirecting internal state through a 2 layer MLP it gets less obvious to me that the world model is really there. I think it would be more impressive if they were only taking “later” activations (further from the input), and using a linear classifier to ensure the world model isn’t in the tile predictor instead of Othello-GPT. I would appreciate it if somebody could illuminate or set my admittedly naive intuitions straight!

That said, I am reminded of another OpenAI paper [1] from way back in 2017 that blew my mind. Unsupervised “predict the next character” training on 82 million amazon reviews, then use the activations to train a linear classifier to predict sentiment. And it turns out they find a single neuron activation is responsible for the bulk of the sentiment!

[1] https://openai.com/blog/unsupervised-sentiment-neuron/

New comment by gtoubassi in "NanoGPT"

gtoubassi — Wed, 11 Jan 2023 17:31:39 +0000

+1. I've benefited greatly from your content, e.g. your CNN lecture was incredibly accessible [0]. I still find transformers stubbornly elude my intuitions despite reading many descriptions. I would very much appreciate your video lecture on this topic.

[0] I think https://www.youtube.com/watch?v=LxfUGhug-iQ

Why Silicon Valley's Optimization Mindset Sets Us Up for Failure

gtoubassi — Sat, 21 May 2022 15:08:19 +0000

Article URL: https://time.com/6096754/silicon-valley-optimization-mindset/

Comments URL: https://news.ycombinator.com/item?id=31458542

Points: 1

# Comments: 0