1. Its a great tool to reduce boilerplate 2. Its great for experimenting with ideas without the overhead that comes with starting a new non trivial project 3. Its great for one offs, demos or anything like that. 4. It helps me to work on some personal side projects that would have never seen the light of day otherwise.

The downsides:

1. As with dynamic languages its a great tool for EXPERT engineers ( not that i am calling me one ) but is often used by Juniors/Entree Level engineers who do not understand the problem, can't tell it exactly what to do, and can't judge the result. And thus it leads to codebases riddled with issues that are hard to find and since they produce a lot of code are a huge liability.

"But look what i made" .... no... no you didn't you don't even understand why its doing something.

e.g we are using Avalonia. Of course everything is drawn in a scalable way, with responsive design etc... Accessibility is built in of course ( with integration with the relevant browser apis ) screenreaders work perfectly as do other accessibility features ) Its not the hap hazard way that flash did this (before there were relevant standards for these features anyways ) Invoking a brower api / interop is easy, the difference is we do not need to compromise our productivity for small things.

The current best option IMO is: Open Full Browser window size canvas (with webgl, webgpu backend graphics ) and draw everyhing yourself ( meaning with something else than the browser layout engine, lots of options available, Flutter, Avalonia etc... ) and deploy with your favourite programming language through WASM.

In fact a next generation browser should bascially be this, with the legacy browser functionality implemented as a WASM module that draws to this single canvas... The browser would become small and much easiert to secure ( only input, audio and general WASI style apis missing and to secure )

The higher level ASP.NET Core stack is also quite efficient and optimized.

BUT: as soon as you gove above the basic middleware pipeline its tends to get bloated and slow. ASP.NET COre MVC is particulary bad.

System.Text.Json is also quite nice, and often is allocation free.

We bascially just us the middleware pipeline and nothing else, and can get millions of requests per second on basic hardware.

Or did they port it during NET/NET Core. As Kestrel has been the recommendation from NET Core 1.0 onwards.

We're using ASP.NET Core Minimal APIS for both API and UI (if configured to run in that mode )

What are the advantages compared to e.g ASP.NET Core Minimal Api ?

(or for example FastEndpoints) ?

1. Microservices are a great tool... IF you have a genuine need for them 2. Decoupling in and on itself with services it not a goal 3. Developers who are bad at writing proper modular code in a monolithic setting will not magically write better code in a Microservice environment.. Rather it will get even worse since APIS ( be it GRPC, Restful or whatever ) are even harder to design 4. Most developers have NO clue about consistency or how to achieve certrain gurantees in a microservice setting ( Fun fact: My first question to developers in that area is: Define your notion of consistency, before be get to the fun stuff like RAFT or PAXOS) 5. You don't have the problems where microservices shine ( e.g banks with a few thounsands RPS ) 6. Your communication overhead will dramatically increase 7. Application A that does not genuinly need microservices will be much cheaper as a monolith with proper code seperation

Right now we have generation of developers and managers who don'T know any better, and just do what everybody else seems to be doing: Microservices and Scrum ... and then wonder why their costs explode.

Most things are settled, but we expect to collect a LOT of data that will be labeled and or auto labeled ( to the tune of 100 MIO video clips )

We will be training multiple models for different tasks from that data and we need a good system to organize it.

Does anybody have any tips experiences with that kind of thing. We can use any on premise or cloud solution....

Specifically we would need

* Data ingestion pipeline ( data will come from field personel ) * Data versioning * Being able to define datasets that are a subset of the whole collected data * Inexpensive storage ( e.g S3 or similar ) * Branching/Merging for maintaining production training data sets * Metadata storage and query capabilities ... * User interface for less tech savy people ( e.g just a git like command line is fine for engineers but not for field personell who are not in IT )

I know of tools like https://dvc.org/ but a) they are just layers on top of git b) break appart on huge datasets without a folder hierarchy ( git tree objects just don't work for linear lists of items ) are only useable by IT personell, and require checking out at least a part of the dataset.

Our datasets would be 100.000.000 x 100 MB = 10 PB of raw data. Training data should be delivered to training nodes via network etc.. we just can't have a full checkout of that data...

Comments URL: https://news.ycombinator.com/item?id=35763004

Points: 1

# Comments: 3

For juniors which have no experience, any sane methodology is better than none, since otherwise you get even more of a mess.

That said, Clean code has some great advice, some mediocre advice and some frankly bad advice, but the authors point are largely irrelevant to 99 % of software engineering.

A really good developer writes clean code using the right abstraction (finding those tends to take the most time and experience) and drop down to a different level of abstraction for high performance areas where it makes sense.

The fact that bad developers suck and write bad code no matter if they use clean code or not does not reflect on the methodology

1. The technology behind it is in the open, and relatively simple ( the models are well understood, and easy to define ) To replicate that ... (with proper engineering etc a good and small team would probably need 1-3 months ) (The model itself could be built in a day )

2. Training: Here comes the biggie, training the above model on lots and lots of data is what gives it its quality. This is the bulk of time and cost, and also why smaller companies have a hard time replicating this. We are talking trainings costs in the 9 figures.

But 2 does not really matter to those giants tech companies, just smaller competitiors.

I would estimate depending on the desired outcome 3-6 months to replicate ChatGPT for those companies.