Hacker News: simicd

Polars 2.0

simicd — Tue, 06 Oct 2026 11:59:01 +0000

Article URL: https://pola.rs/posts/release-polars-2/

Comments URL: https://news.ycombinator.com/item?id=49977177

Points: 409

# Comments: 95

Release of Polars 2.0

simicd — Tue, 06 Oct 2026 11:55:33 +0000

Article URL: https://github.com/pola-rs/polars/releases/tag/py-2.0.0

Comments URL: https://news.ycombinator.com/item?id=49977143

Points: 3

# Comments: 0

Pre-Release of Polars 2.0

simicd — Wed, 02 Sep 2026 14:34:47 +0000

Article URL: https://pola.rs/posts/announcing-polars-2/

Comments URL: https://news.ycombinator.com/item?id=49536922

Points: 8

# Comments: 0

Polars v2.0 Release Candidate

simicd — Wed, 02 Sep 2026 12:09:08 +0000

Article URL: https://github.com/pola-rs/polars/releases/tag/py-2.0.0-rc.1

Comments URL: https://news.ycombinator.com/item?id=49535059

Points: 1

# Comments: 0

New comment by simicd in "Microsoft team creates data-storage system that lasts for millennia"

simicd — Sat, 21 Feb 2026 10:22:55 +0000

At 4.8TB one could add a header section with the full code, instructions how to compile it etc. That would certainly help to reproduce it, assuming civilizations in 10k years still can decypher todays language.

New comment by simicd in "The Business of Betting on Catastrophe"

simicd — Thu, 26 Jun 2025 20:16:59 +0000

It's correct that the number of reinsurers is smaller than that of primary insurers. But the risk born by reinsurers is less correlated, not more. Any given primary insurer has risk clusters (domestic market, line of business, etc.). If a large catastrophe happens in their domestic market they might go bust but what are the chances that it happens simultaneously to all markets globally?

Say you're a primary home insurer in the US. If a hurricane hits you might not have enough capital to rebuild all the homes. A reinsurer which is also covering Europe, Asia, LatAm, etc. is less likely to go bankrupt. The reinsurer can cross-subsidize and use the insurance premiums from other regions to pay out the claims from the US market. All that matters is that on average the loss probabilities and severities are estimated correctly.

And this is just using one line of business as example, reinsurers are covering property, casualty, life and health which add extra layers of diversification.

New comment by simicd in "GitHub Copilot: The Agent Awakens"

simicd — Thu, 06 Feb 2025 18:40:50 +0000

Since it is a fork of VS Code you can install any VS Code extension in Cursor (although manually): https://www.cursor.com/how-to-install-extension

Separating Storage and Compute in DuckDB

simicd — Thu, 26 Dec 2024 13:08:22 +0000

Article URL: https://motherduck.com/blog/separating-storage-compute-duckdb/

Comments URL: https://news.ycombinator.com/item?id=42514958

Points: 3

# Comments: 0

Annotated Logger: A Python package to aid in adding metadata to logs

simicd — Thu, 26 Dec 2024 10:26:17 +0000

Article URL: https://github.blog/developer-skills/programming-languages-and-frameworks/introducing-annotated-logger-a-python-package-to-aid-in-adding-metadata-to-logs/

Comments URL: https://news.ycombinator.com/item?id=42514312

Points: 3

# Comments: 1

New comment by simicd in "Should you ditch Spark for DuckDB or Polars?"

simicd — Sun, 15 Dec 2024 22:10:05 +0000

From what I understood the article refers to the point that DuckDB doesn't provide its own dataframe API, meaning a way to express SQL queries in Python classes/functions.

The link you shared shows how DuckDB can run SQL queries on a pandas dataframe (e.g. `duckdb.query("")`. The SQL query in this case is a string. A dataframe API would allow you to write it completely in Python. An example for this would be polars dataframes (`df.select(pl.col("...").alias("...")).filter(pl.col("...") > x)`).

Dataframe APIs benefit from autocompletion, error handling, syntax highlighting, etc. that the SQL strings wouldn't. Please let me know if I missed something from the blog post you linked!

New comment by simicd in "Sora is here"

simicd — Tue, 10 Dec 2024 20:49:55 +0000

I suspect so, OpenAI is subject to the EU AI Act [0]. Last time they released the Advanced Voice Mode it also took some time before it became available in the EU. Not sure why UK and Switzerland are delayed as well, they are not in the European Union.

[0] https://openai.com/global-affairs/a-primer-on-the-eu-ai-act/

New comment by simicd in "Sora is here"

simicd — Mon, 09 Dec 2024 20:12:31 +0000

Hmm enough capacity for the rest of the world but not EU:

https://help.openai.com/en/articles/10250692-sora-supported-...

New comment by simicd in "Sora is here"

simicd — Mon, 09 Dec 2024 20:10:37 +0000

And in the UK and Switzerland unfortunately

https://help.openai.com/en/articles/10250692-sora-supported-...

New comment by simicd in "Non-elementary group-by aggregations in Polars vs pandas"

simicd — Sat, 16 Nov 2024 08:14:31 +0000

I'm using both Spark and polars, to me the appeal of polars is additionally it is also much faster and easier to set up.

Spark is great if you have large datasets since you can easily scale as you said. But if the dataset is small-ish (<50 million rows) you hit a lower bound in Spark in terms of how fast the job can run. Even if the job is super simple it take 1-2 minutes. Polars on the other hand is almost instantaneous (< 1 second). Doesn't sound like much but to me makes a huge difference when iterating on solutions.

Delta Kernel – Building Delta Lake connectors, made simple (2024)

simicd — Wed, 11 Sep 2024 11:16:42 +0000

Article URL: https://delta.io/blog/delta-kernel/

Comments URL: https://news.ycombinator.com/item?id=41510235

Points: 2

# Comments: 0

Apache DataFusion 40.0.0 Released

simicd — Fri, 26 Jul 2024 01:57:03 +0000

Article URL: https://datafusion.apache.org/blog/2024/07/24/datafusion-40.0.0/

Comments URL: https://news.ycombinator.com/item?id=41075245

Points: 4

# Comments: 0

Apache DataFusion Comet 0.1.0 Release (Apache Spark Accelerator)

simicd — Wed, 24 Jul 2024 02:18:06 +0000

Article URL: https://datafusion.apache.org/blog/2024/07/20/datafusion-comet-0.1.0/

Comments URL: https://news.ycombinator.com/item?id=41053057

Points: 1

# Comments: 0

New comment by simicd in "Announcing Polars 1.0 (Blog Post)"

simicd — Mon, 01 Jul 2024 21:40:49 +0000

Yes only found the announcement [1] that the Polars team and NVIDIA engineers are working on a GPU engine, but other than that no concrete examples. Github issues also don't provide any hints on the status, only one open item [2] where most comments are prior to the announcement.

[1] https://pola.rs/posts/polars-on-gpu/

[2] https://github.com/pola-rs/polars/issues/13111

Announcing Polars 1.0 (Blog Post)

simicd — Mon, 01 Jul 2024 10:56:38 +0000

Article URL: https://pola.rs/posts/announcing-polars-1/

Comments URL: https://news.ycombinator.com/item?id=40844520

Points: 34

# Comments: 7

New comment by simicd in "Htmd: A turndown.js inspired HTML-to-Markdown converter for Rust"

simicd — Sun, 16 Jun 2024 11:56:52 +0000

To add to that, an additional benefit would be you can compile and release it as Python package (Py03/maturin) or compile to WASM so it runs in the browser (with javascript bindings). This makes the code portable while benefiting from Rust's performance/memory safety.