Hacker News: kylebarron

New comment by kylebarron in "Show HN: Parqeye – A CLI tool to visualize and inspect Parquet files"

kylebarron — Tue, 18 Nov 2025 04:14:57 +0000

Looks great!

Another seemingly extremely similar project released in the last few days: https://github.com/raulcd/datanomy

How to Build High-Speed Rail on the Northeast Corridor

kylebarron — Mon, 20 Oct 2025 20:18:03 +0000

Article URL: https://nec.transitcosts.com/

Comments URL: https://news.ycombinator.com/item?id=45648765

Points: 3

# Comments: 0

Making performant Python bindings to compiled code

kylebarron — Tue, 07 Oct 2025 14:31:23 +0000

Article URL: https://kylebarron.dev/blog/fast-python-bindings/

Comments URL: https://news.ycombinator.com/item?id=45503522

Points: 2

# Comments: 0

New comment by kylebarron in "Representing Python notebooks as dataflow graphs"

kylebarron — Sat, 09 Aug 2025 16:29:04 +0000

https://github.com/manzt/juv

New comment by kylebarron in "Notebooks as reusable Python programs"

kylebarron — Wed, 19 Mar 2025 21:22:46 +0000

Agreed, I find this to be a super productive environment, because you get all of vscode's IDE plus the niceties of Jupyter and IPython.

I wrote a small vscode extension that builds upon this to automatically infer code blocks via indentation, so that you don't have to select them manually: [0]

[0]: https://github.com/kylebarron/vscode-jupyter-python

New comment by kylebarron in "Stop using zip codes for geospatial analysis (2019)"

kylebarron — Fri, 07 Feb 2025 20:03:13 +0000

Equal distances to each adjacent neighbor: https://www.uber.com/blog/h3/

Obstore: Efficient streaming object store operations in Python

kylebarron — Thu, 16 Jan 2025 22:46:46 +0000

Article URL: https://github.com/developmentseed/obstore

Comments URL: https://news.ycombinator.com/item?id=42731867

Points: 2

# Comments: 0

New comment by kylebarron in "Railroad Tycoon II"

kylebarron — Tue, 14 Jan 2025 15:33:47 +0000

This worked for me last month on a Mac M2: https://www.portingkit.com/game/278

Literate Flatbush: Understanding a fast, elegant RTree implementation

kylebarron — Wed, 08 Jan 2025 15:47:23 +0000

Article URL: https://kylebarron.dev/literate-flatbush/

Comments URL: https://news.ycombinator.com/item?id=42635401

Points: 1

# Comments: 0

New comment by kylebarron in "Show HN: OpenFreeMap – Open-Source Map Hosting"

kylebarron — Tue, 24 Sep 2024 21:11:10 +0000

https://visgl.github.io/react-map-gl/ is a react interface to MapLibre

New comment by kylebarron in "Show HN: OpenFreeMap – Open-Source Map Hosting"

kylebarron — Tue, 24 Sep 2024 21:09:01 +0000

I develop Lonboard [0], a Python library for plotting large geospatial data. If you have small data (~max 30,000 coordinates), leaflet-based Python libraries like folium and ipyleaflet can be fine, but because Lonboard uses deck.gl for GPU-accelerated rendering, it's 30-50x faster than leaflet for large datasets [1].

[0]: https://developmentseed.org/lonboard/latest/

[1]: https://developmentseed.org/lonboard/latest/how-it-works/

New comment by kylebarron in "Parquet-WASM: Rust-based WebAssembly bindings to read and write Parquet data"

kylebarron — Mon, 22 Apr 2024 20:29:11 +0000

It can read from HTTP urls, but you'd need to manage signing the URLs yourself. On the writing side, it currently writes to an ArrayBuffer, which then you could upload to a server or save on the user's machine.

New comment by kylebarron in "Parquet-WASM: Rust-based WebAssembly bindings to read and write Parquet data"

kylebarron — Mon, 22 Apr 2024 20:25:41 +0000

Arrow JS is just ArrayBuffers underneath. You do want to amortize some operations to avoid unnecessary conversions. I.e. Arrow JS stores strings as UTF-8, but native JS strings are UTF-16 I believe.

Arrow is especially powerful across the WASM <--> JS boundary! In fact, I wrote a library to interpret Arrow from Wasm memory into JS without any copies [0]. (Motivating blog post [1])

[0]: https://github.com/kylebarron/arrow-js-ffi

[1]: https://observablehq.com/@kylebarron/zero-copy-apache-arrow-...

Parquet-WASM: Rust-based WebAssembly bindings to read and write Parquet data

kylebarron — Mon, 22 Apr 2024 15:10:00 +0000

Article URL: https://github.com/kylebarron/parquet-wasm

Comments URL: https://news.ycombinator.com/item?id=40115155

Points: 179

# Comments: 14

New comment by kylebarron in "Friends don't let friends export to CSV"

kylebarron — Wed, 27 Mar 2024 02:36:52 +0000

That's why I'm working on the GeoParquet spec [0]! It gives you both compression-by-default and super fast reads and writes! So it's usually as small as gzipped CSV, if not smaller, while being faster to read and write than GeoPackage.

Try using `GeoDataFrame.to_parquet` and `GeoPandas.read_parquet`

[0]: https://github.com/opengeospatial/geoparquet

New comment by kylebarron in "Friends don't let friends export to CSV"

kylebarron — Wed, 27 Mar 2024 02:26:15 +0000

Your issue is that you're using the default (old) binding to GDAL, based on Fiona [0].

You need to use pyogrio [1], its vectorized counterpart, instead. Make sure you use `engine="pyogrio"` when calling `to_file` [2]. Fiona does a loop in Python, while pyogrio is exclusively compiled. So pyogrio is usually about 10-15x faster than fiona. Soon, in pyogrio version 0.8, it will be another ~2-4x faster than pyogrio is now [3].

[0]: https://github.com/Toblerity/Fiona

[1]: https://github.com/geopandas/pyogrio

[2]: https://geopandas.org/en/stable/docs/reference/api/geopandas...

[3]: https://github.com/geopandas/pyogrio/pull/346

New comment by kylebarron in "Friends don't let friends export to CSV"

kylebarron — Tue, 26 Mar 2024 13:36:27 +0000

Sorry, this is not true _at all_ for geospatial data.

A quick benchmark [0] shows that saving to GeoPackage, FlatGeobuf, and GeoParquet are roughly 10x faster than saving to CSV. Additionally, the CSV is much larger than any other format.

[0]: https://gist.github.com/kylebarron/f632bbf95dbb81c571e4e64cd...

New comment by kylebarron in "Vega-Altair: Declarative Visualization in Python"

kylebarron — Mon, 26 Feb 2024 06:43:22 +0000

Is this open? Is there a way to read more about it? Issues/pull requests?

New comment by kylebarron in "Goodbye, Node.js Buffer"

kylebarron — Tue, 24 Oct 2023 17:52:16 +0000

nodejs-polars is node-specific and uses native FFI. polars can be compiled to Wasm but doesn't yet have a js API out of the box.

As for the fastest way to serialize data to Pandas data to the browser, you should use Parquet; it's the fastest to write on the Python side and read on the JS side, while also being compressed. See https://github.com/kylebarron/parquet-wasm (full disclosure, I wrote this)

New comment by kylebarron in "Protomaps – A free and open source map of the world"

kylebarron — Mon, 23 Oct 2023 13:32:50 +0000

pmtiles are only the counterpart to the "overviews" part of COG. pmtiles doesn't give you analysis-ready data. For that, look to FlatGeobuf or GeoParquet (once it gets a spatial index in v1.1 of the spec)