Hacker News: Smith42

New comment by Smith42 in "US Government directive to suspend access to Fable 5 and Mythos 5"

Smith42 — Sat, 13 Jun 2026 01:33:48 +0000

It's always been this way ever since the first industrial revolution.

New comment by Smith42 in "AI and the Ship of Theseus"

Smith42 — Fri, 06 Mar 2026 15:28:51 +0000

So write it! Shouldn't be much extra to add to the AGPL licence?

Opencode

Smith42 — Mon, 14 Jul 2025 16:29:21 +0000

Article URL: https://opencode.ai/

Comments URL: https://news.ycombinator.com/item?id=44562045

Points: 4

# Comments: 0

US halts student visa appointments and plans expanded social media vetting

Smith42 — Wed, 28 May 2025 08:10:32 +0000

Article URL: https://www.bbc.co.uk/news/articles/cy75eenl46eo

Comments URL: https://news.ycombinator.com/item?id=44113751

Points: 8

# Comments: 3

Mistral Small 3 Announcement

Smith42 — Thu, 30 Jan 2025 14:24:51 +0000

Article URL: https://twitter.com/MistralAI/status/1884968836606136636

Comments URL: https://news.ycombinator.com/item?id=42877950

Points: 15

# Comments: 0

The Multimodal Universe: Enabling Large-Scale ML with 100TB of Astro Data

Smith42 — Wed, 04 Dec 2024 05:36:31 +0000

Article URL: https://arxiv.org/abs/2412.02527

Comments URL: https://news.ycombinator.com/item?id=42314726

Points: 2

# Comments: 0

New comment by Smith42 in "Will we run out of data? Limits of LLM scaling based on human-generated data"

Smith42 — Fri, 07 Jun 2024 17:09:47 +0000

We investigate the potential constraints on LLM scaling posed by the availability of public human-generated text data. We forecast the growing demand for training data based on current trends and estimate the total stock of public human text data. Our findings indicate that if current LLM development trends continue, models will be trained on datasets roughly equal in size to the available stock of public human text data between 2026 and 2032, or slightly earlier if models are overtrained. We explore how progress in language modeling can continue when human-generated text datasets cannot be scaled any further. We argue that synthetic data generation, transfer learning from data-rich domains, and data efficiency improvements might support further progress.

Will we run out of data? Limits of LLM scaling based on human-generated data

Smith42 — Fri, 07 Jun 2024 17:08:29 +0000

Article URL: https://arxiv.org/abs/2211.04325

Comments URL: https://news.ycombinator.com/item?id=40610622

Points: 1

# Comments: 1

New comment by Smith42 in "Astronomy Generates Mountains of Data. That's Perfect for AI – Universe Today"

Smith42 — Tue, 04 Jun 2024 12:52:39 +0000

That really isn't the case, and I am not sure how you could arrive at that unsubstantiated conclusion.

Astronomy Generates Mountains of Data. That's Perfect for AI – Universe Today

Smith42 — Tue, 04 Jun 2024 12:21:01 +0000

Article URL: https://www.universetoday.com/167153/astronomy-generates-mountains-of-data-thats-perfect-for-ai/

Comments URL: https://news.ycombinator.com/item?id=40573725

Points: 2

# Comments: 2

New comment by Smith42 in "AstroPT: Scaling Large Observation Models for Astronomy"

Smith42 — Mon, 27 May 2024 09:21:45 +0000

Abstract:

This work presents AstroPT, an autoregressive pretrained transformer developed with astronomical use-cases in mind. The AstroPT models presented here have been pretrained on 8.6 million 512 × 512 pixel grz-band galaxy postage stamp observations from the DESI Legacy Survey DR8. We train a selection of foundation models of increasing size from 1 million to 2.1 billion parameters, and find that AstroPT follows a similar saturating log-log scaling law to textual models. We also find that the models' performances on downstream tasks as measured by linear probing improves with model size up to the model parameter saturation point. We believe that collaborative community development paves the best route towards realising an open source `Large Observation Model' -- a model trained on data taken from the observational sciences at the scale seen in natural language processing. To this end, we release the source code, weights, and dataset for AstroPT under the MIT license, and invite potential collaborators to join us in collectively building and researching these models.

AstroPT: Scaling Large Observation Models for Astronomy

Smith42 — Mon, 27 May 2024 09:21:45 +0000

Article URL: https://arxiv.org/abs/2405.14930

Comments URL: https://news.ycombinator.com/item?id=40489110

Points: 2

# Comments: 1

New comment by Smith42 in "Building a deep learning rig"

Smith42 — Sat, 24 Feb 2024 14:18:31 +0000

$15k!

New comment by Smith42 in "AnyGPT: Unified Multimodal LLM with Discrete Sequence Modeling"

Smith42 — Wed, 21 Feb 2024 15:56:19 +0000

"Large Observation Model" has a nice ring to it

New comment by Smith42 in "A decoder-only foundation model for time-series forecasting"

Smith42 — Sat, 03 Feb 2024 07:32:37 +0000

If you are interested in this also check out EarthPT, which is also a time series decoding transformer (and has the code and weights released under the MIT licence): https://arxiv.org/abs/2309.07207

New comment by Smith42 in "Sxmo: Linux tiling window manager for phones"

Smith42 — Sat, 27 Jan 2024 13:25:43 +0000

What's new with SXMO? Haven't been keeping up since 2021. Is there a stable phone to run this on now?

New comment by Smith42 in "EarthPT: A time series transformer foundation model"

Smith42 — Fri, 19 Jan 2024 12:44:18 +0000

Wanted to share the code release of EarthPT, a model that predicts future satellite observations in a zero shot setting! I'm the first author so please shoot any questions you have at me.

EarthPT is a 700 million parameter decoding transformer foundation model trained in an autoregressive self-supervised manner and developed specifically with EO use-cases in mind. EarthPT can accurately predict future satellite observations across the 400-2300 nm range well into the future (we found six months!).

The embeddings learnt by EarthPT hold semantically meaningful information and could be exploited for downstream tasks such as highly granular, dynamic land use classification.

The coolest takeaway for me is that EO data provides us with -- in theory -- quadrillions of training tokens. Therefore, if we assume that EarthPT follows neural scaling laws akin to those derived for Large Language Models (LLMs), there is currently no data-imposed limit to scaling EarthPT and other similar ‘Large Observation Models.’(!)

Code: https://github.com/aspiaspace/EarthPT

Paper: https://arxiv.org/abs/2309.07207

EarthPT: A time series transformer foundation model

Smith42 — Fri, 19 Jan 2024 12:44:18 +0000

Article URL: https://github.com/aspiaspace/EarthPT

Comments URL: https://news.ycombinator.com/item?id=39054782

Points: 2

# Comments: 1

New comment by Smith42 in "OpenAI researchers warned board of AI breakthrough ahead of CEO ouster"

Smith42 — Thu, 23 Nov 2023 00:17:06 +0000

Wishful thinkin buddy

New comment by Smith42 in "OpenAI Just Killed an Entire Market in 45 Minutes"

Smith42 — Thu, 16 Nov 2023 16:10:28 +0000

Anyone have a paste of the article? There is a paywall