Comments URL: https://news.ycombinator.com/item?id=31597512

Points: 2

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=30168935

Points: 1

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=28447060

Points: 2

# Comments: 0

Comments URL: https://news.ycombinator.com/item?id=28096802

Points: 3

# Comments: 0

Even if you do a content aware hash where you break the file into chunks and hash each chunk, you still wouldn’t be able to magically recreate the hash of a CP file without also producing part of the CP.

In environments where you’re prioritizing performance I’d still argue streams are likely your best bet when the size of the file to be parsed is not a constant. You wouldn’t want to load 50 large files into ram on a server environment let alone a phone.

If your input buffer is a bunch of tiny 10 KB files and you trust them? Sure, load them into memory and access their indices on the stack. Make sure you reuse the buffer to avoid unnecessary allocations.

If you want parallel processing with zero-allocations then streams with an array pool for their backing buffer are the best bet.

Not loading arbitrary files into memory will always be safer than doing so.

As for decoding - I believe the functions for validating if an array of bytes is an image should be far removed from the decoding and presentation of those bytes to the frame buffer. You don’t need to decode a JPG to validate that a file is a JPG. It either conforms to the standard or it doesn’t; the pixel data is irrelevant.

Don’t load them into memory, parse them as a stream byte-by-byte in accordance with the standard for the codec, check every offset before seeking, and reject images that don’t conform to the standard.

And of course, a ton of fuzzing to accompany it.

We're unleashing the future of computing, collaboration, productivity, and development by using real-time video streaming to change the way that desktop software is built and distributed.

We've opened source all of our code challenges, so if you're interested in any of our open positions you can apply by following the instructions on this page: https://rainway.work/Code-Challenges-f1bc61fecf754d06be253ad...

Enjoy your sad, weird life https://andrew.im/

I love the privilege on display here. You can ignore racial inequality issues because "they are too distracting," meanwhile I have to wake up each morning, read yet another one of my brothers and sisters has been killed by police, and I still have to run my business. Your mediocrity exist in your complacently to the status-quo. On a daily basis we innovate, build, ship, and push tech more than you could ever hope -- and we don't need to ban "political discussions" at work.

If a business chooses to condemn something without backing up that solidarity with meaningful resources and capital, they should be criticized.

- The benchmark code is present in the laboratory directory of the repository.

- We don’t compare to Capt’n Proto because it does not have a stable web-based implementation, at least not one that has the features that make it so fast natively, so there is nothing to compare.

- Flatbuffers are fast but have a notoriously awful API to work with while also creating their own non-standard data structures in languages like C++. Bebop generates standard type-safe code.

- Bebop doesn’t try to compress data other than strings. This is because we don’t want to be responsible for compressing trailing zeroes when faster compression algorithms exist that can be down after encoding. Also most data is tiny.

- Bebop supports discriminated unions and has a much more robust type system than Flatbuffers.

- We’re not convincing anyone to use our stuff. It was made for us and open sourced because it was useful; we don’t need people ripping out their current serializers if there’s no pressure to do so.

Comments URL: https://news.ycombinator.com/item?id=26494340

Points: 2

# Comments: 0

https://andrew.im/essays/a-journey-to-the-american-dream