I'd imagine it's possible bots that have taken root on this site lately - approximately the last year possibly year and a half - seemingly to suppress what could be considered, by sentiment analysis, "negative" news of anything or anyone conservative.

Or those who feel more emboldened to downvote than in previous times on this site; possibly to its growth reaching new demographics. However it feels fairly automated though given how quickly it occurs and the frequency, and how it seemed also to coincide with this last election cycle.

He's fairly lucky he doesn't have a criminal record, but it didn't come without consequences. I think the fact that the sandwich was still wrapped on the ground, hit the officer's shoulder, that the other police at the time were visibly amused during the incident, and clearly joking about it for several days after as well with the officer who had it happen to him, showed that the incident wasn't serious enough to ruin anyone's life over. A formal criminal conviction in the US would've made it hard for him to get employment for some time, if not the rest of his life.

While this affects all models it seems, I think the case gets worse for in particular LLMs because I would imagine all backends, including proprietary ones, are batching users prompts. Other concurrent requests seem to change the output of your request, and then if there is even a one token change to the input or output token, especially on large inputs or outputs, the divergence can compound. Also vLLM's documentation mentions this: https://docs.vllm.ai/en/latest/usage/faq.html

So how does one do benchmarking of AI/ML models and LLMs reliably (lets ignore arguing over the flaws of the metrics themselves, and just the fact that the output for any particular input can diverge given the above). You'd also want to redo evals as soon as any hardware or software stack changes are made to the production environment.

Seems like one needs to setup a highly deterministic backend, by forcing non-deterministic behavior in pytorch and using a backend which doesn't do batching for an initial eval that would allow for troubleshooting and non-variation in output to get a better sense of how consistent the model without the noise of batching and non-deterministic GPU calculations/kernels etc.

However then, for production, when determinism isn't guaranteed because you'd need batching and non-determism for performance, I would think that one would want to do multiple runs in various real-world situations (such as multiple users doing all sorts of different queries at the same time) and do some sort of averaging of the results. But I'm not entirely sure, because I would imagine the types of queries other users are making would then change the results fairly significantly. I'm not sure how much the batching that vLLM does would change the results of the output; but vLLM does say that batching does influence changes in the outputs.

The reforms put in place since then has prevented any rouge sort of activity which might not have been in the interests of the US to peruse, be legal, or otherwise explicitly asked for by the president or congress for them do

Citation #2 is a link a good reads page for "The Lincoln Lawyer" - confused on how this is relevant to "air america"

Citation #3 is not a specific link to any particular article, just the miamiherald's home page.

And someone previously mentioned there is no citation for claim 4

Kind of feels like astroturfing or a bot of some sort. Mind clarifying any of this?