People are also not always very good judges of long-term value. For the longest time the response I'd get whenever I expressed some sentiment of supporting small business, all things being equal, was something along the lines of "well you should support whoever can deliver the good at the lowest price." Although that's true to some extent, in the long term it leads to personal costs as local services are gone, and there's less and less competition. Short-term benefits, long term costs and so forth.

In general, I think there's far too little focus on maintaining healthy competition. The pendulum in public discussions often swings from public to private services and employment, which is a good conversation to have, but there's almost never a serious, critical discussion about increasing competition and what those barriers are. US society is full of things that stifle competition and they don't get 1/10 the attention they deserve. It seems as if when it's brought up, you're forced into these ultra-libertarian or anti-capitalist stereotypes.

The other day I was looking for a cycle restoration shop. There's a great one and they should be thriving right now. But their business was destroyed in riots. Not the business owner's fault. You could make an argument that it's just one more downstream effect of a racist cop killing someone over a counterfeit bill, a very significant one. This effect has played out many many times across the country now I'm sure. But who will foot the bill? The Mpls police dept?

It seems we used to have this intuitive understanding of "bad things happen to good people" and "the community should pitch in collectively to support these things" but it's fading away.

The darwinian model is flawed, unless you're focuses solely on the net benefits to those who benefit, which is circular in its reasoning.

To me, messaging is a mess at the moment, somewhat like IoT because of lack of solid widely adopted standards (either de facto or de jure).

It's extremely difficult to get friends and family to use something. Most decisions are driven by secondary considerations, like it comes with an OS, or as part of an email or office system, or a gaming system. In some cases it's because "it's what everyone is using".

This shifts the threshold a bit in terms of concerns. What I mean by that is given the inertia involved in moving people to use a messaging system, the bar gets raised in terms of moving people off because of network effects. It's hard enough to get any friends or family to use Signal as an alternative to other things; convincing them to switch again introduces other problems.

I'd prefer something that can be used in more decentralized way, but that has its own issues in terms of syncing and always-on problems. And as security increases, more and more inconveniences are introduced -- it might be worth it, but the case still has to be made implicitly or explicitly to friends and family.

Again, not saying these kinds of discussions shouldn't happen, but they often seem kind of theoretical to me or like they're missing the point because of bigger issues with the messaging ecosystem in general. If you're not going to be able to use Signal anyway because everyone you know is using Whatsapp or iMessaging, or feel like messenger use is driven by "whatever is most popular" it feels like it's difficult to weigh things like "won't put on fdroid". I'd love to see it on fdroid but where does that rank?

Science has become so political and subject to hype and fads, at least in biomedicine, that everyone should be deeply skeptical of every claim that is made. That doesn't mean that you should distrust everything completely, just that critical skepticism should be the norm.

If by "science" you mean that "rigorous, logical empirical study" should be part of the decision-making, yes, I agree.

If by "science" you mean that the statements of scientists with prestigious academic appointments should be given priority over other considerations, I say no. Outsider criticism should be welcome, and I think nonscientific considerations are sometimes required.

One of the reasons society is in the mess it is in (at least in the US) is because of a failure to realize that certain segments, like the biomedical-scientific community, are just as driven by human failures as any other.

Whether that's worth it or not to any given person is a personal thing but I was really surprised by it.

I agree a randomized controlled trial would be nice now. But even that is potentially fraught. For example, someone else mentioned the possibility that glucosamine reduces joint pain, which increases mobility, which increases longevity. Randomizing wouldn't really control for that sort of scenario. And that's not even getting into preregistration, meta-analysis etc.

At this point in time, many of the people in power (full professors, administration, etc) see papers as essentially useless, in part because of all the problems being discussed here. p-hacking, guest authorship, etc and so forth and so on. They're seen as a dime a dozen and somewhat ignored.

This might sound good until you understand that this means that they're dismissing the entirety of the scientific dialogue essentially, and that their alternative is grant money. My institution had training workshops for grad students where faculty would tell students that research is essentially worthless unless grant money is attached to it. The idea is that papers are a dime a dozen, and that if something is worthwhile, the feds will put their money where their mouth is so to speak.

The problem with this, of course, is that the grant system is horribly nepotistic and distorted. The biggest predictor of successful grant application last time I looked (based on empirical research in peer-reviewed journals) is co-authoring papers with someone on the review panel. Grant receipt is only weakly related to citation metrics, and from personal experience I can say that institutions are constantly encouraging researchers to inflate grant costs to bring in more indirect costs.

You could fix everything about journals and the publication process and it would do nothing about the shadow scientific world that exists in parallel that drives all the rest of the problems.

Eliminate indirect funds, require tenure of all researchers, set aside funding mechanisms for researchers that aren't tied to specific grant applications, randomize grant rewards, depriortize journals, ... there's a lot of things that need to happen.

When I've talked about this with colleagues, they've argued this would make problems worse, and it might, but I think the idea was more that once you reach a certain threshold you should be funded some amount.

Of course, I'd argue that this is basically the idea behind tenure at an R1 institution, that once you reach a certain level the state or private board of trustees is paying your salary to do research. But nowadays tenure at an R1 institution is proxy for external grant dollars, which defeats the purpose and is redundant.

The real problem is indirect funds, which create profit for institutions. The federal government needs to eliminate indirect funds, so that grant dollars aren't perversely incentivised and tenure is based on research quality rather than profit.

I haven't thought this through yet but it seems interesting and important but also not quite how it's being portrayed in the media, and not quite as significant in some ways.

I could write a book about this stuff. The stories I could tell about what's behind those "outstanding academic track records"...

The problem, if anything, is trying to apply a business model to academics, equating research quality with federal grant dollars, taking away real intellectual freedom protections, and then ignoring all the ponzi scheming and exploitation that occurs. Everyone has their heads in the sand, knows academics (at least biomedical research) is full of BS, and just goes on pretending like it's not because no one knows of a good alternative, or doesn't have the courage or power to change things.

What's funny [sad?] to me is that your description of PMs sounds exactly like the most credentialed, accomplished researchers I know on paper.

It's interesting to me regarding some of the examples in the linked piece. Ghostwriting reviews, for example, is actually seen as a good practice in a lot of circles because it provides experience to grad students with the review process. Those guest authorships? Very grey area between that and collaborative authorships. It's not the grunt work, it's the idea, right? Or is it that ideas are a dime a dozen, and actually doing the work is important? I can't tell which it is anymore -- it seems to depend on what benefits those in power.

Someone else posted something about how 1% of research is fraud, and 80% is bad. I think the percent of fraud is probably higher, the percent bad research is lower, and the difference is much more fuzzy than you'd think initially. The really difficult thing is that tiny incremental contributions is how things actually work. No one wants to admit this though. Bad research is actively incentivized, and there's credit bubbles everywhere.

The worst problem is that this credentialing bubble is everywhere with everything, as another posted noted. The problem isn't the credentialing per se, it's how it's detached from reality, the real demands of the tasks. Having a credential doesn't mean that the person is competent for all the tasks it nominally encompasses; conversely, those tasks don't necessarily require the credential that's often demanded.

At the time the story in this article begins, administrators at many places were completely unfamiliar with the general idea behind open content distribution. In such places, online systems for course content were seen as a way of locking down the intellectual property that courses represent, not to distribute it. Where I was at, faculty were explicitly admonished not to distribute any online course material because it was seen as leaking university IP. Course management systems were seen as a way to deliver materials online securely so as to ensure that they weren't widely distributed. Even if someone wanted to put the effort into something that might now be called a MOOC, it was doubly frowned upon because you were putting effort into teaching rather than research (especially if you were untenured), and triply so because you were putting effort into distributing courses online, which was a fringe activity.

By the time MOOCs came about and got a lot of press, and many of us were sort of shaking our heads at what seemed inevitable, it was a little too little too late. In online space, where social factors like reputation become amplified 100-fold, it's difficult to compete or add to the reputational weight of places like MIT or Harvard. Adding to this was the constant monetizing mindset, of trying to lock down materials, not recognizing the advertising/popularizing role that open content distribution enables. Then too was a sort of "so what" feeling, because in the absence of something like a pandemic, who really cares, from the perspective of the administration -- students come to campus and that's where the dollars come from; MOOCs and online course distribution of whatever form were seen as experimental, the realm of the extension service which is valued but only to a point.

There's lots of issues the piece doesn't get into, or only gets into in a half-hearted way, which is how these types of online teaching dynamics really mess with the traditional structures. So instead of having an interesting conversation about how unfamiliar it is for universities to be dealing with Course A at Institution 1 competing with Course B at Institution 2, it recapitulates the discussion in terms of Malan, which misses the bigger picture entirely. These issues are not about one instructor of one course at one university.

Maybe I'm just getting jaded, but I feel like this piece does a disservice to the behemoth of problems at universities represented by the online instructional debacle we've witnessed. Like lots of things, the pandemic is exposing widespread systemic problems, not creating them.

As far as I can tell though, functionally this is breaking down. People can find preprints and archived papers, and do, if they're searching by topic.

So journals at this point are providing a peer review portal, and formatting. I happen to think the formatting does provide value. My sense is that at a good journal, there's a kind of stochastic improvement in errors and formatting, so that the numbers of errors go down on average, and the formatting improved, on average, over iterations back and forth with the copyeditor.

As for peer review, I'm not so sure anymore. My sense is that it does provide some kind of stamp of approval from experts in the area, so if you don't know much about an area, it provides some sense that at least some small group of people in the area believe it meets some kind of basic standards. But that says very little, and the amount of noise in the review process is large.

I think the core of the academic communication system is slowly being hollowed out, and being replaced by blogs, things like twitter and mastodon, and archives. At this point the peer review journal process provides some value, but it's being propped up by tradition. Already, with COVID, we're increasingly seeing the focus on preprints. Journalists and others are careful to note something hasn't been peer-reviewed yet, but everyone knows it matters little because they can turn to experts to find out what they think of it.

If there aren't formal attempts to create an alternative, I think we'll just be left with people posting and passing around preprints and discussing them on twitter, mastodon, blogs, and message groups. If people want the nice formatting, and some stamp of approval, I think something else will have to be worked out. But the journals are starting to feel like they're getting in the way, in general, and represents some kind of power or status structure more than quality control system.

Paying reviewers I think creates bad incentives as the author of the post points out. So do author-pays systems. What is maybe missing from the piece is some recognition that in the past, reviewers reviewed and editors edited in part as part of their job. That is, you were paid as a faculty member at a university, and that was what people understood you did. Pre-internet, this was all valuable service. Now that universities and others are more focused on faculty bringing in profits rather than paying for their services -- and questions are being raised about the value of journals in general -- we are seeing these questions about what reviewers get paid.

I think in the future there will be value in article hosting and searching, and providing website frameworks for discussion and peer review, but I'm not sure they will look like journals per se. You'll see things like arxiv.org, but with commentary, rating, discussion, and approval infrastructure over them. That's what large libraries and research centers will be donating money to or paying for. I think journals per se will eventually start to seem kind of stodgy and old fashioned.

DD seems like voodoo to me. Research has shown that DL models can reproduce even random noise training data perfectly. So DD seems like an assertion that somehow an overparameterized model can contain more information about a source than a sample dataset, like "free information." It seems like it violates the data processing inequality or something.

Is there just confusion about this phenomenon in general, or are there good resources on this?

Comments URL: https://news.ycombinator.com/item?id=23739387

Points: 1

# Comments: 0

First, there's a sort of framing issue with "subjective answers to questions" per the press release. These subjective experiences are the very nature of the thing of interest: feeling desperate, crying for help, loss of energy, and so forth. To dismiss them is, at some level, is akin to a chemist dismissing oxygen gas as an appropriate target of study because it's all subatomic particles at some level. These types of press releases often start with this notion, in my mind, because it's necessary to create some sense that the study in question is revolutionary.

I do think they're something of a bootstrapping problem with supervised approaches, which is why in psychopathology and psychiatry research there's a trend toward modeling symptom/behavioral presentations using "unsupervised" approaches. This is partially the impetus behind NIH RDoC, for example, and when you do this, you end up with similar but not identical constructs as DSM diagnoses. The recent DSM (DSM-5) was supposed to address this in part but got mired in politics.

If you look at the actual results, by far the most important features in predicting future symptom state are previous values of those symptoms. So follow-up symptom A is best predicted by baseline symptom A. This autocorrelation importance is a sort of law of behavioral individual differences. The EEG variables are adding in the prediction, but relatively weakly, which further speaks to the central importance of the subjective variables, which is informationally mediating whatever is going on under the skull.

This is really further underscored if you start thinking about things like this: are the EEG signals adding most to prediction of "physical" symptom improvement? There's also a literature about severity of symptoms; it's possible to make predictions of symptom response based on other criteria from previous studies and this paper does little to address that in a head-to-head comparison. K-fold cross validation is also good, but isn't the same as replicating something on entirely different sets of people with different sorts of heterogeneity due to unknown sources.

It's an interesting paper but people need to be cautions about the hype that goes on in these areas (biological psychiatry and AI).

The problem in both regions, if you want to call it a problem, is that much of the variety is in local cheeses you'll never or rarely see going outside of a zip code, state, or country (in the case of Europe). So someone will move to, say, Virginia in the US and conclude that they've sampled US cheese, not being aware of all the local cheeses in another state that never travel outside of the area of a few counties. The same thing can happen in the UK and Europe. The cheeses I've loved most have been from these local dairies, cheeses that don't quite fit into traditional categories, and that I can never find unless I'm physically in the area.

I'm actually not so sure it's that common relative to the number of people who actually leave. After I left my position I started learning about other tenured faculty who left their positions also, some of whom I respected quite a bit and who I never imagined would have any reason to leave their position. One place interviewed me for a faculty position later, the primary problem being that they had multiple tenured faculty leave and had holes to fill. None of these individuals who left their positions wrote about it.

I do think you're right in the sense that it's this thing that no one knows what to do with. You've built up these relationships, your colleagues are all in academics, and many of them are happy. So they don't understand. You're kind of a glitch in the matrix, and no one knows how to approach it because it's not supposed to happen. I've been told everything from that I'm a total badass to crazy and insane. I myself haven't figured out which is more accurate yet. Maybe both.

The fact that these experiences compel some kind of cathartic explanation speaks volumes about the problems in academics though. It's like there's the commonly agreed-upon story in academics, the face being presented, and then there's reality, and they don't line up, so people just kind of pretend the glitches don't exist.

Not too long ago on HN there was a blog piece going through one person's (an economist?) exploration of whether or not biomedical research was in crisis. (I've tried to find it but can't.) It was frustrating to me because so much of what they detailed was true, but in the end they just decided something like "oh well, it's obvious this field is in crisis, but research is getting done in the end so it doesn't matter." It's like this ship in flames, where people are burning and there's emergency sirens going off, and as long as the individuals in power are happy, and there's a good narrative being presented to the public, with results in the media to talk about, no one cares.

I just can't tell where this is headed: if the ship is going to finally sink so far that it becomes something you can't hide, or if it's just going to keep on like this indefinitely, or if it's going to get fixed, or what. Universities have survived much worse than current circumstances, so it seems implausible to me anything will fundamentally change.

I don't necessarily think it has to be that way, and I don't necessarily think it's the best way, certainly not for everyone, but I think for a lot of people it provides one way. Right now I think it's a broken path, but I'm not sure what other paths there are to replace it at the moment, given that employers (for the most part, maybe not entirely) seem to be abandoning that role.

To me the bigger underlying subtext in a lot of these discussions is the vacuum that currently exists in terms of providing a safety net for people as they find their way. I think this could be provided by employers or something but I really don't think it is at the moment; colleges might also be flawed in being too expensive or not really delivering on their promises or something, and maybe we're at an inflection point. But if you think of spaces where society is ok with people figuring things out, or learning, or not really being fully developed yet, college is a major place. Outside of that it increasingly feels like people are treated as machines, interchangeable and replaceable, with little to no sense of potential or anything of that sort.

Believe me, I feel like universities are very broken at the moment in many many ways. I just don't really see them going away in the near future, even if they are remade, and I think there's a good opportunity for society to really remake them in a way that's more useful, whatever that is. It may involve restructuring what a college education entails, or paying for it with public funds, or whatever, but that opportunity is there.

In speaking with many students and colleagues, one thing has struck me, which is that many of the current undergrads are enrolled because they want to get away from home. They want some true transition to adulthood, where they live on their own and develop into an independent adult with a vocation and adult identity. This is maybe what I've heard articulated here and elsewhere about being exposed to new ideas, etc. but maybe isn't quite the same. It's something I never really quite appreciated the importance of until recently, but looms large in their minds. Many of them, even though they're taking courses "off-campus" and online, are still in their college town living in their apartment, and so forth. They didn't go home, they just stayed put.

The analogy with books and music is telling because for awhile there too there was a lot of hype about how the internet would lead to some radical reorganization and disruption of those industries. And in many ways it did. But many of the power inequities, the monopolies, and so forth are the same, and the fundamentals are the same. We've migrated a lot from radio and physical media to streaming services, but the players are still there. I suspect something similar will happen with universities; we'll just see a lot more diversity in what form an undergraduate education actually looks like. For some it will involve 4 years in a dorm / on-campus setting. For others it might be more eclectic. There are also some things that just can't really be duplicated at home: an undergrad isn't going to go out and purchase a mass spectrometer for the most part, or PCR equipment, or have storage room full of embalmed anatomy specimens to dissect.

One thing that's important to be mindful of is that this on-campus experience many of us remember is in many ways evaporating anyway. I don't mean to say that's ok, or acceptable, or whatever, but sometimes we talk about the experience of being on campus and discussing things with others in person, forgetting that a major societal issue right now is how that that used to be the norm for everything and is increasingly not. So we have these chats online, on HN comment sections, or over IM, or videoconferencing anyway, even if you're not in college. Whatever might be lost from the on-campus experience is being lost everywhere to some extent; I point this out only to suggest that if we're ok with it in some domains, why should college be special? Or is college the canary that we should be paying closer attention to?

I agree with the perspective of the author of the article that something fundamental needs to change, but my guess is 10 years from now everything will look very different and the same at the same time. I also think there's many problems with universities and higher education that have nothing to do with the cost-benefit calculus in some abstracted sense, but rather to do with how it should be paid for, whether universities are funded well enough, whether they're getting those funds from the right places, whether administration is too top-heavy, whether employers and society abuse interpretation of degrees, and so forth and so on.