Comments URL: https://news.ycombinator.com/item?id=46232321

Points: 7

# Comments: 3

First bias is "perception", which is influenced by how media want to communicate and who own them - in France, mostly right wing billionaires. There is no actual correlation with actual, real, statistics.

Second, by who answer the survey - often people concerned about insecurity, which is a topic from the right.

Finally, there is no control on how many times you can vote, and some people demonstrated that with very few knowledge, you can completely change the results by sending thousands of vote [1].

The fact that Nantes is deemed highly insecure in France is also the consequence of this city being socialist and the place where police killed a guy named Steve during a party. So these attacks on Nantes being dangerous can also be interpreted as a backlash[2].

Please Hacker News, you're better than this, don't fall in this trap...

[1]: https://xcancel.com/dbertho/status/1574761634840592384 [2]: https://www.index.ngo/en/news/steve-maia-canico-trial-of-pol...

It seems the first "grid scale batteries" were derived from EV batteries, and are planned for 1 or 2 decades[1].

Basically, we are discussing battery ageing here, which is a complex problem[2].

According to the different studies on the topic I found, mentioning specifically "large-scale" installation like the ones discussed here, the answer is definitely and deceptively the same: between 10 and 20 years[3][5]. More precisely.

From [3]:

> To address the global effort to decrease carbon emissions, many consumers, corporations, and energy providers are adopting the use of electric vehicles and stationary energy storage systems paired with renewable electricity generation. These systems often utilize large-format lithium-ion batteries [...]. Real-world battery lifetime is evaluated by simulating residential energy storage and commercial frequency containment reserve systems in several U.S. climate regions. Predicted lifetime across cell types varies from 7 years to 20+ years, though all cells are predicted to have at least 10 year life in certain conditions.

From [5]:

> In the 2020 report, calendar life for both LFP and NMC Li-ion systems was stated as 10 years. The 2022 report takes additional information from long-term laboratory work (Saft, 2021) and product data into account (Baxter, 2021b) to establish new calendar lives of 16 years for LFP and 13 years for NMC. The calendar life is unchanged for 2030.

I also claim that battery are not renewable. One might argue that, if we can recycle batteries like we recycle regular glass, it could be considered as renewable. However, today there are 2 industrialized processes that are not satisfying (pyrometallurgical and hydrometallurgical processing) which "require high energy, and/or complex wet-chemistry steps"[4]. Some explored processes called "direct recycling"[4], which also has severe drawbacks but at least is more promising.

Which makes me think: we are, at least, making huge bets on the future here, as we risk 1) having huge amount of aged batteries in 1 or 2 decades, 2) no more mineral resources to extract.

[1]: https://www.quora.com/How-long-do-grid-storage-batteries-las...

[2]: https://cea.hal.science/cea-01791260/document

[3]: https://www.sciencedirect.com/science/article/abs/pii/S23521...

[4]: https://www.sciencedirect.com/science/article/pii/S2352152X2...

[5]: https://www.pnnl.gov/sites/default/files/media/file/ESGC%20C...

Don't get me wrong: the article is very interesting. I really learnt something: I discovered "system inertia", I was not aware of stability issues linked to inverters, and did not know about grid-forming & grid-following inverters, and the research about finding the minimal amount of grid-forming to keep a power-grid stable in case of an issue in a given power plant. All of these topics are very interesting.

But making a connection between inverters and ecology through the "net zero" terms seems either off topic, misleading or irrelevant. First because this "net zero" term is heavily criticized as it means carbon are still emitted but companies are paying for carbon credits (that are not compensating at all the carbon emitted for many reasons [1]). Here building solar panels, wind turbines & batteries emits CO2, and their lifespan is relatively short (at most 10 years for batteries, ~25 years for wind turbines & solar panels, compared to hundreds of years for a dam[7]). Second because climate change is not the only concern about ecology, there are concerning questions about mineral resource extraction, like lithium[2] that is heavily used in batteries, but more generally, we are already extracting the whole Mendeleev periodic table[3]: we don't have alternative mineral resources for batteries or other technologies, the only solution is to extract, produce & consume less. Third, if your only goal is to reduce carbon dioxide equivalent (eqCO2), you should advertise nuclear power plant as the solution. Depending on studies, they produce the same amount or less eqCO2 compared to a wind turbine without batteries[4]. Of course, often eqCO2 is not the only important subject here (being renewable/sustainable is also important, and uranium is a limited resource). And finally, the fact we use renewable energy more and more did not lead to a worlwide energy transition, but an addition. Having a transition will require way more than technologies[5], something that is also not discussed here.

Speaking about solutions to pack a higher percentage of Intermittent renewable energy sources (IRES)[6] in a power-grid through the help of batteries and inverters would have been more accurate in my opinion. Maybe "Why we were not able to achieve 100% renewable energy before?" if you want to be catchy, and it's not perfect, as you are still hiding that you rely on lot of batteries, that are far from being renewable.

As a conclusion, I would say it would be great to be careful when engineers (here IEEE) discuss specific technologies (here power-grid inverters) to not draw conclusion too quickly (having a positive environmental impact), as it's far from being obvious. I know they want to be read, I know that the title must be catchy to attract readers, but it's not an excuse as illustrated above.

[1]: https://demandclimatejustice.org/wp-content/uploads/2020/10/...

[2]: https://www.cnbc.com/2023/08/29/a-worldwide-lithium-shortage...

[3]: https://www.euchems.eu/euchems-periodic-table/

[4]: https://www.edfenergy.com/media-centre/news-releases/over-it...

[5]: https://www.sciencedirect.com/science/article/abs/pii/S22146...

[6]: https://en.wikipedia.org/wiki/Variable_renewable_energy

[7]: https://www.power-technology.com/data-insights/power-plant-p...

  A Suggested-by: tag indicates that the patch idea is suggested by the person named and ensures credit to the person for the idea.
  Please note that this tag should not be added without the reporter's permission, especially if the idea was not posted in a public forum. 
  That said, if we diligently credit our idea reporters, they will, hopefully, be inspired to help us again in the future.

Ref: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/lin...

My idea when I installed Haiku was to make my own version of the "old computer challenge"[1], with an emphasis on using GUI apps.

Similarly to @probono (a FOSS dev), I also found Haiku "shockingly good"[2] at being a lightweight, responsive, easy-to-use desktop OS.

After some patching, I was even able to compile Tectonic[3], a modern LaTeX engine written in Rust, and Quaternion a Matrix client supporting E2EE[4]. All that running on a single core Athlon 64 and 1.5GB of RAM.

I posted some screenshots in a Mastodon threads if you are curious[5] (but my posts are in french sorry :/). And of course this comment is posted from Haiku!

[1]: https://dataswamp.org/~solene/2021-07-07-old-computer-challe...

[2]: https://medium.com/@probonopd/my-first-day-with-haiku-shocki...

[3]: https://tectonic-typesetting.github.io/en-US/

[4]: https://github.com/quotient-im/Quaternion

[5]: https://mastodon.tedomum.net/@tgoldoin/109554115997967651

I think an interesting comparison, for both Linen and Matrix, would be to compare these 2 approaches: Linen natively indexed conversation and this Matrix "static" client. I would be especially interested by what additional features Linen provides in term of indexing compared to this static client.

[1]: https://github.com/matrix-org/matrix-static [2]: https://view.matrix.org/

I also work on a Rust IMAP server that is far from being as feature complete as yours. I also chose your `mail-parser` library to parse RFC822/5822, but we observed that in many cases, we did not have enough information to build some BODY/BODYSTRUCTURE responses. We also discovered that line count and many details are not very obvious on IMAP, did you run some tests to compare your IMAP server outputs to existing servers? Or, more generally, what is your approach to ensure compatibility / integration with the existing email ecosystem?

In any case, congratulation for your project, we will follow it closely! I experimented how big these protocols became with all their extensions, this is an impressive work!

[1]: https://github.com/Ekleog/kannader

[0]: https://rclone.org/crypt/

[1]: https://nextcloud.com/endtoend/

On the plus side, it survived Hacker News Hug of Death. Indeed, the website we linked is hosted on our own Garage cluster made of old Lenovo ThinkCentre M83 (with Intel Pentium G3420 and 8GB of RAM) and the cluster seems fine. We also host more than 100k objects in our Matrix (a chat service) bucket.

On the minus side, this is the first time we have so much coverage, so our software has not yet been tested by thousands of people. It is possible that in the near future, some edge cases we never triggered are reported. This is the reason why most people wait that an application reaches a certain level of adoption before using it, in other words they don't want to pay "the early adopter cost".

In the end, it's up to you :-)

Due to our design choice, you can add and remove nodes without any constraint on number of nodes and size of the storage. So you do not have to overprovision your cluster as recommended by MinIO[0].

Additionally, and we planned a full blog post on this subject, adding or removing a node in the cluster does not lead to a full rebalance of the cluster. To understand why, I must explain how it works traditionally and how we improved on existing work.

When you initialize the cluster, we split the cluster in partitions, then assign partitions to nodes (see Maglev[1]). Later, based on their hash, we will store data in its corresponding partition. When a node is added or removed, traditional approaches rerun the whole algorithm and comes with a totally different partition assignation. Instead, we try to compute a new partition distribution that minimize partitions assignment change, which in the end minimize the number of partitions moved.

On the drawback side, Garage does not implement erasure coding (as it also the reason of many MinIO's limitations) and duplicate data 3 times which is less efficient. Garage also implements less S3 endpoints than Minio (for example we do not support versioning), the full list is available in our documentation[2].

[0]: https://docs.min.io/minio/baremetal/installation/deploy-mini...

[1]: https://www.usenix.org/conference/nsdi16/technical-sessions/...

[2]: https://garagehq.deuxfleurs.fr/documentation/reference-manua...

When failures occur, repair is done through workers that says when they launch, when they repair chunks, and when they exit in the logs. We also have `garage status` and `garage stats`. The first command displays healthy and non healthy nodes, the second one displays the queue length of our tables and chunks, if their values are greater than zero, we are repairing the cluster. We are documenting failure recovery in our documentation: https://garagehq.deuxfleurs.fr/documentation/cookbook/recove...

For the near future, we plan to integrate opentelemetry. But we are still discussing the design and information we want to track and report. We are currently discussing these questions in our issue tracker: https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/111 https://git.deuxfleurs.fr/Deuxfleurs/garage/issues/207

If you have some knowledge/experience on this subject, feel free to share it in these issues.

I just deleted the AAAA entry for this machine. In the meantime, if the result is cached for you, you can pass the `-4` argument to force IPv4:

  git clone -4 git@git.deuxfleurs.fr:Deuxfleurs/garage.git 

For a Raft-based object store, your gateway will receive the write request and forward it to the leader (+ 100ms, 2 messages). Then, the leader will forward in parallel this write to the 9 nodes of the cluster and wait that a majority answers (+ 100ms, 18 messages). Then the leader will confirm the write to all the cluster and wait for a majority again (+ 100ms, 18 messages). Finally, it will answer to your gateway (already counted in the first step). In the end, our write took 300ms and generated 38 messages over the cluster.

Another critical point with Raft is that your writes do not scale: they all have to go through your leader. So on the writes point of view, it is not very different from having a single server.

For a DynamoDB-like object store (Riak CS, Pithos, Openstack Swift, Garage), the gateway receives the request and know directly on which nodes it must store the writes. For Garage, we choose to store every writes on 3 different nodes. So the gateway sends the write request to the 3 nodes and waits that at least 2 nodes confirm the write (+ 100ms, 6 messages). In the end, our write took 100ms, generated 6 messages over the cluster, and the number of writes is not dependent on the number of (raft) nodes in the cluster.

With this model, we can still provide always up to date values. When performing a read request, we also query the 3 nodes that must contain the data and wait for 2 of them. Because we have 3 nodes, wrote at least on 2 of them, and read on 2 of them, we will necessarily get the last value. This algorithm is discussed in Amazon's DynamoDB paper[0].

I reasoned in a model where there is no bandwidth, no CPU limit, no contention at all. In real systems, these limits apply, and we think that's another argument in favor of Garage :-)

[0]: https://dl.acm.org/doi/abs/10.1145/1323293.1294281

> Why not using Riak and adding an S3 API around it.

Riak was developed by a company named Basho that went bankrupt some years ago, the software is not developed anymore. In fact, we do no need to add an S3 API around Riak KV, Basho even released "Riak Cloud Storage"[0] that exactly does this: provide an S3 API on top of Riak KV architecture. We plan to release a comparison between Garage and Riak CS, Garage has some interesting features that Riak CS does not have! In practice, implementing an object store on top of a DynamoDB-like KV store is not that straightforward. For example, Exoscale, a cloud provider went this way for their first implementation of their KV store, Pithos[1], but rewrote it later as you need special logic to handle your chunks (they did not publish Pithos v2).

> Most apps don't have S3 support

We are maintaining in our documentation an "integration" section listing all the compatible applications. Garage already works with Matrix, Mastodon, Peertube, Nextcloud, Restic (an alternative to Borg), Hugo and Publii (a static site generator with a GUI). These applications are only a fraction of all existing applications, but our software is targeted at its users/hosters.

> A distributed system is not necessarily highly available

I will not fight on the wording: we come from an academic background where the term "distributed computing" has a specific meaning that may differ outside. In our field, we define models where we study systems made of processes that can crash. Depending on your algorithms and the properties you want, you can prove that your system will work despite some crashes. We want to build software on these academic foundations. This is also the reason we put "Standing on the shoulders of giants" on our front page and linking to research papers. To put it in a nutshell, one critic we address to other software is that sometimes they lack theoretical/academic foundations that lead to unexpected failures/more work to sysadmins. But on the theoretical point, Basho and Riak were exemplary and a model for us!

[0]: https://docs.riak.com/riak/cs/2.1.1/index.html [1]: https://github.com/exoscale/pithos

- Garage is easier to deploy and to operate: you don't have to manage independent components like the filer, the volume manager, the master, etc. It also seems that a bucket must be pinned to a volume server on SeaweedFS. In Garage, all buckets are spread on the whole cluster. So you do not have to worry that your bucket fills one of your volume server.

- Garage works better in presence of crashes: I would be very interested by a deep analysis of Seaweed "automatic master failover". They use Raft, I suppose either by running an healthcheck every second which lead to data loss on a crash, or sending a request for each transaction, which creates a huge bottleneck in their design.

- Better scalability: because there is no special node, there is no bottlenecks. I suppose that with SeaweedFS, all the requests have to pass through the master. We do not have such limitations.

As a conclusion, we choose a radically different design with Garage. We plan to do a more in-depth comparison in the future, but even today, I can say that if we implement the same API, our radically different designs lead to radically different properties and trade-off.

The goal is to say, as an individual:

  - I am not ok anymore that so much sensitive data are collected
  - I know data collection had negative impacts on individuals and society      
  - I can, and we should live without collecting so much data    
  - Individuals and society should come before companies