Or the author's newsletter: https://www.construction-physics.com/

https://en.wikipedia.org/wiki/Pixel_shift

EDIT: Sigma also has "Foveon" sensors that do not have the filter and instead stacks multiple sensors (for different wavelengths) at each pixel.

https://en.wikipedia.org/wiki/Foveon_X3_sensor

Could you also get d2 into GitHub and Notion while you have it here :)

Having to figure out the exact pixel widths defeats the point of these tools, at least for me.

Which links to this blog post explaining the choice in more detail: https://smallcultfollowing.com/babysteps/blog/2022/04/12/imp...

Comments URL: https://news.ycombinator.com/item?id=43297605

Points: 5

# Comments: 0

[1]: https://bookshop.org/info/ebooks ("Can I read my ebooks on my Kindle, Kobo, Nook, etc.?")

You can sort of see what benefits you might get from a post like this, though: https://willayd.com/leveraging-the-adbc-driver-in-analytics-...

While we're not using Arrow on the wire here, the ADBC driver uses Postgres's binary format (which is still row oriented) + COPY and can get significant speedups compared to other Postgres drivers.

The other thing might be to consider whether you can just dump to Parquet files or something like that and bypass the database entirely (maybe using Iceberg as well).

Arrow Flight SQL defines defines a full protocol designed to support JDBC/ODBC-like APIs but using columnar, Arrow data transfer for performance (why take your data and transpose it twice?)

https://arrow.apache.org/blog/2022/02/16/introducing-arrow-f...

There's an Apache-licensed JDBC driver that talks the Flight SQL protocol (i.e. it's a driver for _any_ server that implements the protocol): https://arrow.apache.org/blog/2022/11/01/arrow-flight-sql-jd...

(There's also an ODBC driver, but at the moment it's GPL - the developers are working on upstreaming it and rewriting the GPL bits. And yes, this means that you're still transposing your data, but it turns out that transferring your data in columnar format can still be faster - see https://www.vldb.org/pvldb/vol10/p1022-muehleisen.pdf)

There's an experiment to put Flight SQL in front of PostgreSQL: https://arrow.apache.org/blog/2023/09/13/flight-sql-postgres...

There's also ADBC; where Flight SQL is a generic protocol (akin to TDS or how many projects implement the PostgreSQL wire protocol), ADBC is a generic API (akin to JDBC/ODBC in that it abstracts the protocol layer/database, but it again uses Arrow data): https://arrow.apache.org/blog/2023/01/05/introducing-arrow-a...

I think the difference is more that DataFusion is built as a library so you can plug it into the product you're building (e.g. Comet, which plugs it into Spark, or pg_lakehouse, which plugs it into Postgres). Polars could be used that way, but it's also a functional package you can pip install and use as a Pandas alternative right now.

Coincidentally I was looking into C++ documentation generators again.

In terms of integration, what I've settled on for apache/arrow-adbc is using Sphinx as the toplevel site generator, then writing a script that generates fake Intersphinx indices for a Doxygen site. That way you can link to Doxygen items from within Sphinx without having to hardcode URLs, instead by referencing a class name or similar, and Sphinx will warn if you reference something nonexistent, without having to use something like breathe that tries to render the Doxygen output within Sphinx. (Same approach with Javadoc -> Sphinx, too.)