It's a chip (and associated hardware) that can do linear algebra operations really fast. XLA and TPUs were co-designed, so as long as what you are doing is expressible in XLA's HLO language (https://openxla.org/xla/operation_semantics), the TPU can run it, and in many cases run it very efficiently. TPUs have different scaling properties than GPUs (think sparser but much larger communication), no graphics hardware inside them (no shader hardware, no raytracing hardware, etc), and a different control flow regime ("single-threaded" with very-wide SIMD primitives, as opposed to massively-multithreaded GPUs).

In essence, one is not telling the model "This. This is what you should output next time." but rather "I liked this reply. Have a cookie." The behaviors that you can learn in RL are more subtle, but you get a lot less information per step. That's because, in a causal language modeling objective, when I tell you "For the prompt X, you should output exactly Y[0...m)", you get a gradient for P(Y[0] | X), another one for P(Y[1] | X Y[0..1)), another for P(Y[2] | X Y[0..2)), another for P(Y[3] | X Y[0..3)), and so on. It's a lot more of a step-by-step guidance, than it is a sentence-wise reward that you get in the RL framework. In RL, I'd give you a cookie for P(Y | X). What part of Y made me give you that cookie? Was there even such a part? Was it perhaps some internal representation that made everything in Y better? That's for the model to learn.

The poster you're replying to seems to have more of an emphasis in how the "Wrong." parent was worded. There was no need to be that confrontational, when one is adding a point of information ("Z does Y, with Z != X") that is most likely consistent with the thread title, and with the article content.

  * There's almost always a simple geometric intuition, and low-dimensional intuition can get you quite far even in high dimensional cases.
  * You can surprisingly often get by with closing your eyes and saying "my problem is linear" three times. See: All of neural networks.
  * Linear problems have practically all nice properties you could ever ask of any function.

I don't know what the G stands for. Possibly luminosity of some kind?

Comments URL: https://news.ycombinator.com/item?id=29937844

Points: 87

# Comments: 16

Given a functor m of Hask, we define a squiggly arrow ~>, where a ~> b is a -> m b. If these ~> are the arrows in some category (which we call the Kleisli category for m), then we call m a monad. Here id :: a ~> a in that category is what we call return :: a -> m a, and the composition (.) :: (b ~> c) -> (a ~> b) -> (a ~> c) in that category is used to create bind :: m a -> (a -> m b) -> m b, where bind x f = (.) f (const x) (return ()), where (.) is the squiggly arrow (.) we just mentioned. Bind is what's spelled >>= in Haskell, and is what's behind the "x <- f" do-notation.