Originally for churches, my draft article below describes how this problem affects all individuals and institutions. I recommend solutions which include AllSides.com (amazing!) and search engines for retrieving news from multiple outlets. I have a prototype. Progress is slow on my tool because I work two jobs with my free time mostly going to ministry serving Christ and others.

https://heswithjesus.com/mediabias.html

I haven't finished reviewing and adding Drooid yet. I'll still link it because it's a good idea:

https://apps.apple.com/us/app/drooid-news-from-all-sides/id6...

https://play.google.com/store/apps/details?id=social.drooid&...

(Note: I'm not affiliated with or paid by any of these companies. I am a paying supported of AllSides because I believe they'll do a lot of good.)

I'm hoping the good results with AI models drive down the prices of traditional tools. Then, we can train open models to integrate with them.

Their redundancy architecture is interesting. I'd be curious of what innovations went into rad-hard fabrication, too. Sandia Secure Processor (aka Score) was a neat example of rad-hard, secure processors.

Their simulation systems might be helpful for others, too. We've seen more interest in that from FoundationDB to TigerBeetle.

It's designed for drafting legal documents for lawyers. It's pretrained on a ton of court documents.

re why generalists are better

Much knowledge we have builds on prior knowledge. The prior knowledge is often reused across domains. Analogous reasoning, important in creativity, also connects facts or heuristics across different domains. Also, just being better at English.

If training a coding LLM, it needs to understand English, any concepts you type in, intrinsic knowledge about your problems, general heuristics for problem solving, and code with has comments and issues. The comments and issues might contain or need any of the above.

That's why I believe generalist LLM's further trained on code work better than LLM's trained only on code.

Python can do the algorithms. It's quick to develop and debug. There's tons of existing code in data science and ML fields. It's worse in the other areas I mentioned, though.

So, a transpiler that generated Rust or safe C/C++ from legacy and AI-generated Python could be a potent combination. What do you think about that?

https://www.kl3m.ai/

Author is correct that intelligence is compounding. That's why domain-specific models are usually general models converted to domain-specific models by continued pretraining. Even general models, like H20's, have been improved by constraining them to domain-supporting, general knowledge in a second phase of pretraining. But, they're eventually domain specific.

Outside LLM's, I think most models are domain-specific: genetics, stock prices, ECG/EKG scans, transmission shifying, seismic, climate, etc. LLM's trying to do everything are an exception to the rule that most ML is domain-specific.

Windows, its default, used so much memory that there was not much left for apps.

Ubuntu used 500MB less than Windows in system monitor. I think it was still 1GB or more. It also appeared to run more slowly than it used to on older hardware.

Lubuntu used hundreds of MB less than Ubuntu. It could still run the same apps but had less features in UI (eg search). It ran lightening fast with more, simultaneous apps.

(Note: That laptop's Wifi card wouldn't work with any Linux using any technique I tried. Sadly, I had to ditch it.)

I also had Lubuntu on a 10+ year old Thinkpad with an i7 (2nd gen). It's been my daily machine for a long time. The newer, USB installers wouldn't work with it. While I can't recall the specifics, I finally found a way to load an Ubuntu-like interface or Ubuntu itself through the Lubuntu tech. It's now much slower but still lighter than default Ubuntu or Windows.

(Note: Lubuntu was much lighter and faster on a refurbished Dell laptop I tested it on, too.)

God blessed me recently by a person who outright gave me an Acer Nitro with a RTX and Windows. My next step is to figure out the safest way to dual boot Windows 11 and Linux for machine learning without destroying the existing filesystem or overshrinking it.

For example, I used to get verbatim quotes and answers from copyrighted works when I used GPT-3.5. That's what clued me in to the copyright problem. Whereas, the smallest models often produced nonsense about the same topics. Because small models often produce nonsense.

You might need to do a new test each time to avoid your old ones being scraped into the training sets. Maybe a new one for each model produced after your last one. Totally unrelated to the last one, too.

Examples: KLEE test generator; combinatorial or path-bases testing; CPAChecker; race detectors for concurrency; SIF information flow control; symbolic execution; Why3 verifier which commercial tools already build on.

If translated to C or Java, we can use decades worth of tools for static analysis and test generation. While in Python and Javascript, it's easier to analyze and live debug by humans.

Multiple wins if the translators can be built.

That doesn't even account for lightweight, formal methods. SPARK Ada, Jahob verification system with its many solvers, Design ny Contract, LLM's spitting this stuff out from human descriptions, type systems like Rust's, etc. Speed run (AI) producing those with unsafe stuff checked by the combo of tools I already described.

Also, like ForAllSecure's Mayhem, I think they can differentiate on automatic patching that's reliable and secure. Maybe test generation, too, that does full coverage. They become drive by verification and validation specialists who also fix your stuff for you.

"School-type long essays only seem to exist in academia."

Does an AI know what an essay is? Would it consider any long, descriptive post an essay? Especially if pretraining data has many people describing long posts as essays or "essay-like?" Or only actual essays? And what is an actual essay again?

I think AI's might have different interpretations due to the above questions. They might also conflate essays with longer, detailed, or argumentative posts. We'd have to put a bunch of posts into a bunch of AI's to ask how they classify them.

1. They reward short or under-developed essays. I'd say most online content, especially with high upvotes next to the post, fits that. Social media surely does.

2. If it's longer posts, the system starts nitpicking it on minor details, like grammar. We see this even on Hacker News, a community valuing quality, with some longer submissions. It's also a debate tactic to derail opponents' better arguments in many discussions which are in their pretraining data.

3. Essays with more praise get higher scores and with more criticism get lower scores. "Get on the Bandwagon" Effect. Echo chambers. One person writes a thing followed by 5-20 people confirming it. That's probably in the pretraining data. It might survive some filtering/cleaning strategies, too.

So, no, I think these AI's are acting way too human. They need to fine-tune them to act like more, reasonable humans. That will initially take RLHF data for many types of situations. Given pretraining bias, they might also have to train them to drop the bad habits the article mentions.

I wonder how much independent review went into that step.

People would probably upvote that.

Non-believers are much more vulnerable to demonic activity than believers. There's also a goal where distracting them from Christ is all they nerd to stay on the road to Hell. So, UFO sightings should be much higher in areas with non-believers than areas with Christians. He shared his data here:

https://web.archive.org/web/20220521004104/https://www.godan...

(Note: I haven't peer reviewed his methodology.)