(And yes, a lot of science is software. Analysis is software.)

The issue isn't a lack of economic interest.

It might be a lack of training data in addition to inherent complexity, but it's certainly not a lack of economic interest.

But yeah, if you want to feed it math and get code, it's reasonably okay with that. All LLMs I've used seem bad at understanding things that don't look like broad human knowledge. I've seen this same general issue across many different models. (And to be fair, geology, geophysics, and remote sensing are what I'm testing, and their semi-rare niches.)

It's also quite dangerous because it's not obvious that what it's doing is complete hallucinations unless you actually are a domain expert. Things _sound_ reasonable. E.g. "this is likely feature X" which _does_ exist, but is absolutely _not_ relevant to the problem or present in the input dataset.

But my current employer is pushing this exact thing (human language + scientific data + LLM -> advanced analysis of scientific data by LLM -> business decisions) and it _really_ worries me. It often gives the rough equivalent of "Start the procedure by severing the patient's aorta. Once they stop moving, you can deal with the hangnail". Just in very reasonable sounding language. And a lot of people don't know any better, because most users aren't domain experts.

Comments URL: https://news.ycombinator.com/item?id=47234119

Points: 3

# Comments: 0

I've done a fair bit in the field, but a huge part of my career has been mining old datasets and reinterpreting things in light of new data/etc.

What the article is describing isn't new in any way. But it also doesn't remove the need for fieldwork or the need for the experience of having done fieldwork to use existing datasets. Observational sciences (e.g. geology, biology, etc) where you can't easily replicate the environment you are studying in the lab are always going to hinge on some sort of fieldwork.

Finding creative ways to use existing data doesn't change that.

Sure, the pieces average 6 faces when materials are relatively homogenous and iostropic (i.e. no preferential direction to break in and no free surface nearby). However, as they note in the article, this isn't always the case. Things like mud flats and other cases with very anisotropic materials and/or free surfaces nearby don't fracture with the same average.

This is a good example of a potential metric that could be used to give some clues about overall material behavior even if all you have are the broken remains.

Fractal dimension is also pretty esoteric. However, it's somewhat widely used in geoscience, even though what we're measuring isn't _actually_ fractal. It's still a very useful comparative metric, though, because it lets us measure how complex an interface or surface is quantitatively and scale-independent.

With that said, the simple truth of it is that we know next to nothing about these ecosystems and really can't accurately estimate impacts. They're quite possibly significant, but we just don't have much info to go off of and studies like this are sorely needed.

The article makes this point, but it's relatively far in and I felt it was worth making again.

With that said, my employer now appears to be in this business, so I guess if there's money there, we can build the satellites. (Note: opinions my own) I just don't see how it makes sense from a practical technical perspective.

Space is a much harder place to run datacenters.

Typing in python within the scientific world isn't ever used to check types. It's _strictly_ only documentation.

Yes, MyPy and whatnot exist, but not meaningfully. You literally can't use them for anything in this domain (they wont' run any of the code in question).

Types (in this subset of python) are 100% about documentation, 0% about enforcement.

We're setting up a _documentation_ system that can't express the core things it needs to. That worries me. Setting up type _checks_ is a completely different thing and not at all the goal.

"array-like" has real meaning in the python world and lots of things operate in that world. A very common need in libraries is indicating that things expect something that's either a numpy array or a subclass of one or something that's _convertible_ into a numpy array. That last part is key. E.g. nested lists. Or something with the __array__ interface.

In addition to dimensionality that part doesn't translate well.

And regardless, if the type representation is not standardized across multiple libraries (i.e. in core numpy), there's little value to it.

There's not a good way to say "Expects a 3D array-like" (i.e. something convertible into an array with at least 3 dimensions). Similarly, things like "At least 2 dimensional" or similar just aren't expressible in the type system and potentially could be. You wind up relying on docstrings. Personally, I think typing in docstrings is great. At least for me, IDE (vim) hinting/autocompletion/etc all work already with standard docstrings and strictly typed interpreters are a completely moot point for most scientific computing. What happens in practice is that you have the real info in the docstring and a type "stub" for typing. However, at the point that all of the relevant information about the expected type is going to have to be the docstring, is the additional typing really adding anything?

In short, I'd love to see the ability to indicate expected dimensionality or dimensionality of operation in typing of numpy arrays.

But with that said, I worry that typing for these use cases adds relatively little functionality at the significant expense of readability.

I wish I'd done this more.

In some cases there was no way to. For example, we once woke up to find that the European half of our team had been laid off as part of huge cuts that weren't announced and even our manager had no idea were coming. There's no good way to do layoffs, but I think that "sudden shock" approach is worst of all, personally. You don't get to say goodbye in any way and people don't get to plan for contingencies at all. (The other extreme of knowing it's coming for a year and applying for your own job and then having 2 months to sit around after you didn't get it also sucks, and I've done that as well. You can at least make plans in that case, though.)

On the other hand, in a _lot_ of other cases, you do have a chance to say goodbye. Take it. This is really excellent advice. It's worth saying something, at very least to the people you really did enjoy working with.

There's a decent chance you work with some of those folks in the future, and even if you don't, it really does mean something to be a kind human.