An auto-encoder is trained on [activation] -AV-> [text] -AR-> [activation], where [activation] belongs to one layer in the LLM model M.

Architecture.:

    Model being analyzed (M): >|||||>  
    Auto-Verbalizer (AV) same as M, with tokens for activation: >|||||>  
    Auto-Reconstructor (AR) truncated up to the layer being analyzed: ||>

The AR is trained on a simple reconstruction loss.

The AV is trained using an RL objective of reconstruction loss with a KL penalty to keep the verbalizations similar to the initial weights (to maintain linguistic fluency).

- Authors acknowledge, and expect, confabulations in verbalizations: factually incorrect or unsubstantiated statements. But, the internal thought we seek is itself, by definition, unsubstantiated. How can we tell if it is not duplicitous?

- They test this on a layer 2/3 deep into the models. I wonder how shallow and deep abstractions affect thought verbalization?

    [1]: https://github.com/ddclient/ddclient  
    [2]: https://kb.netgear.com/1058/What-is-Dynamic-DNS-DDNS

                 ____.----.____
          ______/              \______
    _____/                            \_____
    ________________________________________

    (simulations)  (real world data)  (simulations)

We started with physics-based simulators for training policies. Then put them in the real world using modular perception/prediction/planning systems. Once enough data was collected, we went back to making simulators. This time, they're physics "informed" deep learning models.

    https://iahmed.me

Github Pages deployment.

Here's my first website from when I was in college and had no experience in web dev. I still keep it on for nostalgia:

    https://iahmed.me/old_www/

For example

- how can I reliably have a decision block to end the loop (or keep it running)?

- how can I reliably call tools with the right schema?

- how can I reliably summarize context / excise noise from the conversation?

Perhaps, as the models get better, they'll approach some threshold where my worries just go away. However, I can't quantify that threshold myself and that leaves a cloud of uncertainty hanging over any agentic loops I build.

Perhaps I should accept that it's a feature and not a bug? :)

- For LLM-assisted output, the more complex the LLM-writing is, the less likely the paper is to be published. From eyeballing, at WC=-30, both have similar chances of publication (~46%). At the upper range of WC=25, LLM-assisted papers are ~17% less likely to be published.

- LLM-assisted authors produced more preprints (+36%).

I wonder:

- What is the distribution of writing complexity?

  * Does the 17% publication deficit at WC=25 correspond to 17% of the 36% excess LLM-assisted papers being WC=25, thus nullifying the effect? Although, it puts extra strain on the review process.

Comments URL: https://news.ycombinator.com/item?id=46507149

Points: 2

# Comments: 0

Back in grad school, I was out making new friends. I was playing tennis 4-5 times a week. I'd invite players for post-game coffees (in the morning) and dinner (evenings) at every game. Consistency mattered. I'd ask every time. Slowly we had our regulars. Our coffee times became an institution in and of themselves.

People are busy, yes. But, people also want to be in demand. People also don't want to be rejected. And, people also don't want to be left out.

Asking around, I was exposing myself to rejection. Some folks appreciated their time being demanded. More still joined because they didn't want to be left out.

A light-weight speculative model adapts to usage, keeping the acceptance rate for the static heavy-weight model within acceptable bounds.

Do they adapt with LoRAs?

I agree, cubsats are a wonderful way, for college students even, to tinker with space(-adjacent) tech.

https://github.com/hazrmard/SatTrack

Features:

  - Local. No internet connection needed.  
  - Manual. Every transaction is added by the user.
  - One-off or arbitrarily recurring transactions.  
  - No lock-in. Check out your data any time.  
  - Arbitrary metrics to track performance. 
  - Hosting on the cloud for mobile access. 

I've been using Google sheets + forms for the last 8 years to track my finances. It's worked well, except for minor inconveniences. This app is my answer to my own problems.

What have we discovered (and made available as treatment) for advanced breast cancers?

[1]: https://news.ycombinator.com/item?id=44219379 [2]: https://pmc.ncbi.nlm.nih.gov/articles/PMC8990477/

Comments URL: https://news.ycombinator.com/item?id=44244450

Points: 28

# Comments: 10

Having done research in RL, a big problem with incremental research was to reproduce comparative works, and to validate my own contributions. A simple library like this, with built-in tools for visualization and a gridworld sandbox where I can validate just by observation, is very helpful!

Shoutout to Alif, a finetune of Llama 3 8b on Urdu datasets: https://huggingface.co/large-traversaal/Alif-1.0-8B-Instruct

It'd be great to see a comparison.

[1]: https://github.com/lululxvi/deepxde [2]: https://github.com/nvidia/modulus

* Manufacturing: Computer vision to pick anomalies off the assembly line.

* Operation: Accelerometers/temperature sensors w/ frequency analysis to detect onset of faults (prognostics / diagnostics) and do predictive maintenance.

* Sales: Timeseries analyses on numbers / support calls to detect up/downticks in cashflows, customer satisfaction etc.

Until I discovered Hamiltonian physics :)

Modelica is acausal. You define the variables and how they are related (equations). The compiler handles variable dependencies and resolution internally.

There are pros & cons of each. Both are used for simulating cyber-physical systems.