> Ubuntu priority

> High

For anyone interested in playing around with it, it requires python 3.11, else it will segfault.

According to the code the conversation turn index is a signal that enters so it is judging on something that does not transfer to other tasks. In the sampling it moved the score more than the actual content.

"give me all the specs in a list" -> "which architectural changes would simplify the implementation based of these specs" -> delete everything -> "implement the new specs"

What would be strange are hackers that are fascist. Fascism demands surrender to power and obedience, which is antithetical to the hacker sprit. Questioning systems, equalizing power imbalances is the hacker spirit.

With Bitnami discontinuing their offer, we recently switched to other providers. For some we are using a helm chart and this new offer provides some helm charts but for some software just the image. I would be interested to give this a try but e.g. the python image only various '(dev)' images while the guide mentions the non-dev images. So this requires some planning.

EDIT: Digging deeper, I notice it requires a PAT and a PAT is bound to a personal account. I guess you need the enterprise offering for organisation support. I am not going to waste my time to contact them for an enterprise offer for a small start-up. What is the use case for CVE hardened images that you cannot properly run in an CICD and only on your dev machine? Are there companies that need to follow compliance rules or need this security guarantee but don't have CICD in place?

There are some different approaches here and there which come from making it application specific, e.g. we added a periodic reconciliation check. I also built a debouncer into the queue to give special treatment to burst in the load.

[1] https://blog.benediktsvogler.com/blog/building-a-distributed...

Gameplay wise this is super basic and has been done endless times before. It looks like a "hello world" of games, so maybe focus on bringing either quality or novelty.

This project came out of a need while working on an internal ML-based SaaS (which didn't pan out). Initially, we used the RunPod API directly in the application, as RunPod had the most affordable GPU pricing at the time. But I also had a GPU server at home and wanted to run experiments even cheaper. Since I had good experiences with Kubernetes jobs (for CPU workloads), I installed k3s and made the home GPU node part of the cluster. The idea was simple: use the local GPU when possible, and burst to RunPod when needed. The app logic would stay clean. Kubernetes would handle the infrastructure decisions. Ideally, the same infra would scale from dev experiments to production workloads.

What Didn't Work My first attempt was a custom controller written in Go, monitoring jobs and scheduling them on RunPod. I avoided CRDs to stay compatible with the native Job API. Go was the natural choice given its strong Kubernetes ecosystem. The problem with the approach was that when overwriting pod values and creating virtual pods, this approach fought the Kubernetes scheduler constantly. Reconciliation with runpod and failed jobs lead to problems like loops. I also considered queuing stalled jobs and triggering scale-out logic, which increased the complexity further, but it became a mess. I wrote thousands of lines of Go and never got it stable.

What worked The proper way to do this is with the virtual kubelet. I used the CNCF sandbox project virtual-kubelet (https://github.com/virtual-kubelet/virtual-kubelet), which registers as a node in the cluster. Then the normal scheduler can use taints, tolerations, and node selectors to place pods. When a pod is placed on the virtual node, the controller provisions it using a third-party API, in this case, RunPod's.

Current Status It's source-available under a non-commercial license for now — I'd love to turn this into something sustainable.

I'm not affiliated with RunPod. I shared the project with RunPod, and their Head of Engineering reached out to discuss potential collaboration. We had an initial meeting, and there was interest in continuing the conversation. They asked to schedule a follow-up, but I didn't hear back to my follow ups. These things happen, people get busy or priorities shift. Regardless, I'm glad the project sparked interest and I'm open to revisiting it in the future.

Happy to answer questions or take feedback. Also open to contributors or potential use cases I haven't considered.

Comments URL: https://news.ycombinator.com/item?id=44168045

Points: 3

# Comments: 0