Some examples:

* Once you've learned a few chip pinouts, you can pretty much guess unknown pinouts just from identifying a few ground/control/address pins, as even chip-on-board globs follow similar layouts [1]. However, despite plenty of datasheet archives being publicly available, none of them allow you to actually search by pin function [2], so you potentially have to go through dozens of datasheets of similar model ids to find what you need.

* UART baud rates that are likely used are in the single digits, they can be easily bruteforced.

* JTAG pins you need to interface with can go up to a dozen or so, there are enumeration scripts you can run in an Arduino to identify which pin has which function. These scripts also identify the IDCODE which you can lookup against boundary scan files if you need so [3]. But in most cases, you will interface with JTAG without thinking of the state machine behind it.

* Reverse engineering memory maps is a matter of following data read/write patterns and inferring associated functionality. You will bump into several address cross-references that also hint at what are the base addresses of each map. It's a more general skill you develop as you go, and Ghidra's decompilation made it much more accessible in the last years. The author went with a elaborate linker script but a more bare-bones approach would be to link code as a distinct ELF object, then copy its text section over to offset 0x20A0000-0x2010000 in the firmware image, and patch the initializers.

* Soldering and associated skills can also be self-learned from tutorials, pick several videos and learn the tricks/mistakes each of them cover.

So, in practice? Each of these does not require a vast amount of knowledge for things to happen, even allowing one to skip required reading of huge bibles that are recommended to electronics beginners. This is how a lifetime gets reduced to a few months of non-working hours.

When getting into hardware hacking, what I felt was the main blocker is how a lot is described at a superficial level, without enough breadcrumbs one can follow to reproduce the same results. Sure, the pictures of spaghetti wires and decapped chips look awesome, but nobody learns from that. Unlike the software side where you are given the source and everything you need to lookup is in front of you.

[1]: https://qufb.gitlab.io/writeups/mysteries

[2]: https://github.com/qufb/PinoutDB

[3]: https://bsdl.info/index.htm

It looks like the compilation database is only used for discovering source files, without any additional cross translation unit analysis [1]. Even if you don't plan on implementing data flow tracking or other passes, it can still be interesting to know on which translation unit a function declaration is defined on.

Also, it seems we always have to recompile each unit when switching between source files. Consider pre-compiling and caching so it's only done once. That should also enable users to search across files/ASTs.

[1]: https://clang.llvm.org/docs/analyzer/user-docs/CrossTranslat...

It's also not all program-integrity, which is the only work that could justify such blanket statements. Worse is better as long as it addresses the problem domain.

Fintech clients are generally interested in knowing whether a transaction happening _right now_ is fraud. They want to know that in a few milliseconds, for high-dimensional data. It's work done at a scale where relational databases cannot meet these real-time constraints, and instead find other uses like historical data loading. That's how you end up with in-memory databases, stream-processing engines, and yes, even machine learning.

Having said that, some of the author's points are valid, and I'm looking forward for their next writings, in particular dealing with noisy alerts is a general problem beyond performance engineering.

> Ghidra supports the 8051 architecture but not code banking.

Usually in these ISAs an I/O port or a register sets the bank number, so any processor module should be able to resolve concrete banked references. But you still need to know what that register holds in various code paths, which are likely dynamically computing those values.

No tooling can give this out-of-the-box, as it relies on knowing the concrete initial state of the system (i.e. memory and register contents), and knowing what to return when hooking into I/O accesses.

Once these are known, we can leverage the built-in pcode emulator and run it with this state. It seems nowadays Ghidra has some built-in support for Z3, but I personally never used it, so I'm not sure how viable it is for symbolic execution. Regardless, with either approach, we would now have concrete banked code references being resolved, and could script some auto annotation of the disassembly with these references. These would be equivalent to what the author gathered from the logic analyzer trace.

A pure static analysis approach seems to suggest one would manually brute-force through all possible bank numbers at any given code path, which I guess is only viable if you have the time for that.

It seems like it was a follow-up from previous bruteforce efforts, which include a spreadsheet with various results, but it would help to have some conclusions on which were best: http://forum.redump.org/topic/51851/dumping-dvds-raw-an-ongo...

Also, couldn't find any source/download for DiscImageMender.

The later model Pixter Multimedia had the full memory space accessible via JTAG, which is how some carts and even boot ROM got dumped a while ago [1], is it the same deal with Pixter Color?

That OpenOCD script was a bit flaky, and sometimes the boot ROM would be already unloaded before reading, maybe you have some insights in how to make it more robust.

btw, have you looked into the original Pixter? The cart connector seems to have a very narrow bus, so it doesn't look like those carts have code, and probably can only be dumped with a decap.

[1]: https://qufb.gitlab.io/writeups/pixter

For other architectures, it feels like a missed opportunity to not have an independent WASM build of MAME's debugger, as the whole project could already be built in WASM (although I think the latest versions were broken, as that target isn't actively maintained): https://docs.mamedev.org/initialsetup/compilingmame.html#ems...

> Things may be non-byte-aligned bitstreams.

* https://doc.kaitai.io/user_guide.html#_bit_sized_integers

> Arrays of structures that go "read until id is 5, but if id is 5, nothing else of the structure is emitted."

* https://doc.kaitai.io/user_guide.html#_repetitions

> Fields that may be optional if some parent of the current record has some weird value.

* https://doc.kaitai.io/user_guide.html#do-nothing

> Files may be composed of records at arbitrary, random offsets that essentially require seeking to make any sense of it.

* https://doc.kaitai.io/user_guide.html#_relative_positioning

> The metadata of your structure may depend on some early parameter (for example, is this field big-endian or little-endian?)

* https://doc.kaitai.io/user_guide.html#param-types

* https://doc.kaitai.io/user_guide.html#switch-advanced

Regarding the Kafka issues pointed out:

* 2 only affect you if you are using transactions (and I would be interested in knowing of alternatives that better handle exactly-once semantics);

* Consumer.close() is likely off the critical path of your applications (personally I never encountered these hangs);

* Aborted reads and torn transactions were mentioned as being fixed by KIP-890, of which the mentioned ongoing work was completed since that article was published: https://issues.apache.org/jira/browse/KAFKA-14402

* Rust9x: Compile Rust code for Windows 95, NT and above: https://news.ycombinator.com/item?id=31112273

* Rust on MIPS64 Windows NT 4.0: https://news.ycombinator.com/item?id=29240174

A lot of the times, it just happens that someone was the first person that even bothered trying digging into the code. Specially after decompilation became much more accessible for less popular architectures with Ghidra. Give it a try, there's plenty of low hanging fruit! I've submitted another case some time ago.

Also luckily, considering other OS easter eggs, it doesn't seem like there was any obfuscation involved, like "chained xor stored in bitmap resource of badly supported executable format": https://x.com/mswin_bat/status/1504788425525719043

I'll give a specific example with Apache Spark: AWS provides a managed cluster via EMR. You can configure your task nodes (i.e. instances that run the bulk of your submitted jobs to Spark) to be autoscaled. If these jobs fetch data from managed databases, you might have RDS configured with autoscaling read replicas to support higher volume queries.

What I've frequently see happening: tasks fail because the task node instances were downscaled at the end of the job, because they are no longer consuming enough resources to stay up, but the tasks themselves haven't finished. Or tasks failed because database connections were suddenly cut off, since RDS read replicas were no longer transmitting enough data to stay up.

The workaround is to have a fixed number of instances up, and pay the costs you were trying to avoid in the first place.

Or you could have an autoscaling mechanism that is aware of your application state, which is what k8s enables.

But if you do want some visual feedback like LEDs and such, you would have to write a layout and input logic for that (AFAIK has been done for MIDI synths).

It would also be more appropriate for shellcode / nostdlib experiments, although it's not clear how well CPUlator handles dynamic linking / relocations in their ELF executable support (seems like they build on their server [2], but run in a custom interpreter?)

[1]: https://qufb.gitlab.io/writeups/2023/02/26/Put-A-Sonic-On-It... [2]: https://cpulator.01xz.net/doc/#compiling

Comments URL: https://news.ycombinator.com/item?id=36231371

Points: 2

# Comments: 0

Instead, I've found much more deliberate practice and therefore value from reverse engineering: just like in debugging, you want to understand the underlying logic through program analysis.

You can pick security CTFs, crackmes, malware samples, proprietary software with bugs you want to fix, or functionalities you want to add, or even games where you want to find some hidden content, extract some resources, or modify some behaviors. In all that you will find something of your interest, and apply approaches that I think translate well to software development:

* Differential tracing: Want to know how some action reflects in the codebase? Take k instruction traces where you do everything except that action, then take a trace where you do that action, find out what are the unique differences in that last trace. Want to know what data gets written? Same approach with memory dumps and breakpoints. How do different inputs affect these changes? You will learn to be methodic and throughout in what you test and log.

* Recognizing patterns: Sometimes you don't have symbols for your functions, are you able to identify printf at a glance, or will you waste time following the logic of the function? Do you see relative offsets being used and recognize accesses to an array? With source code all this happens on a more macro view, such as algorithms or design patterns hidden in all those coupled functions and classes. But the micro view also applies: figure out what constants relate to specific functionalities, and you can grep your way to relevant functions or documentation.

* Avoiding boilerplate: This follows from the previous point, since you want to recognize the flow of data through the codebase, in order to have some call hierarchy to follow, otherwise it's easy to waste time on functionally that is irrelevant to you. Start with how data enters the application: stdin, files, database connections, http endpoints... Tests, examples, or client apps will also help here.

Oh and don't worry about learning some assembly language, just make that investment, since that's the straightforward, well defined, predictable part.

As an aside, I didn't find the writeup to be overly descriptive. The author does a great job linking every step and how they came to each conclusion along the way, even including hypotheses where they came out empty-handed. When a writeup omits these, it makes the author look like they were somehow enlightened to find the solution, instead of just following a methodical approach.