Comments URL: https://news.ycombinator.com/item?id=46301893

Points: 1

# Comments: 0

But in this case, I tried in earnest to use nextjs for a project with auth & stripe, etc. this past week, and I can't believe how frustrating it is to get stupid things like modal dialogs to work properly in the client.

I have tons of experience with React SPAs. But the client/server divide in Next remains quite inscrutable to me to the extent that I'm just going to start again with Django (where I nearly started it in the first place).

So yes, it doesn't work like I expect it to either...

A few thoughts:

1. You can't cut off the embedding layer or discard the tokenizer without throwing out the model you're starting with. The attention matrices are applied to and trained with the token embedding layer.

2. Basically the same thing regarding the tokenizer. If you need to add some tokens, that can be done (or you can repurpose existing tokens) if your script is unique (a problem I face periodically). But if you are initializing weights for new tokens, that means those tokens are untrained. So if you do that for all your data, you're training a new model.

3. The Gemma model series sounds like a good fit for your use case. I'm not confident about Hebrew support, let alone Hasidic Yiddish, but it is relatively multilingual (more so than many other open models). Being multilingual means that the odds are greater than they have tokens relevant to your corpus that have been trained towards an optimal point for your dataset.

4. If you can generate synthetic data with synonyms or POS tags, then great. But this is a language model, so you need to think how you can usefully teach it natural sequences of text (not how to tag nouns or identify synonyms - I also did a bunch of classic NLP, and it's depressing how irrelevant all that work is these days). I suspect that repurposing this data will not be worth it. So, if anything, I'd recommend doing that as a second pass.

5. Take a look at unsloth notebooks for training a gemma3 model and load up your data. I reckon it'll surprise you how effective these models are...

I've been dreaming of a set of lego-style bits of a mouse that can be assembled together... want another button? here you go. Want it on the side? Modify the 3D print file. Want bluetooth? Use this board... Want USB-C? Use that board... Want both? We've got you covered... Want a hyper-scroll wheel? Well, Logitech has a patent on that one, but here's the closest thing you can get on a DIY mouse. Now click these buttons in the configurator and hit "upload", and the firmware is installed to use your new mouse on any machine.

One thing that worked strikingly well was translation on non-Indo-European languages. Like I had success with Thai and Bahasa Indonesian -> English...

In this case, I changed the prompt to:

---

Tallest mountains (in order):

```

- Mount Everest

- Mount K2

- Mount Sahel

- Mount Fuji

- Mount McKinley

```

What is the second tallest mountain?

---

Suddenly, it got the answer right 95+% of the time

Still actually falling behind the official scores for o3 high. https://aider.chat/docs/leaderboards/

Comments URL: https://news.ycombinator.com/item?id=44193328

Points: 349

# Comments: 230

Comments URL: https://news.ycombinator.com/item?id=43794638

Points: 1

# Comments: 3

Comments URL: https://news.ycombinator.com/item?id=43686768

Points: 1

# Comments: 0

My current tag line is "JS with guardrails, without footguns"

https://chicory-lang.github.io/

https://github.com/chicory-lang/compiler

Comments URL: https://news.ycombinator.com/item?id=43448265

Points: 4

# Comments: 0