Not exactly your usecase, but a useful one nevertheless.

https://www.graphiti.dev

I quickly noticed that they had employed lazy loading, which would have made that all but impossible. It took me a good few minutes to realise that if they had lazy loading, there had to be a backend, and I was overjoyed when I found out it was serving JSON.

All in all, it was probably much cheaper for them to have me hitting the API endpoint every minute than scraping the website even once a day

This worked as there were only ever 30-50 cats online at one time. If it was a thousand, I'm not sure what I would have done.

Edit: I realise now this was a rhetorical question. Oops!

The primary reason was to learn Elixir, so this was just a well-timed excuse to explore the language (and Phoenix, the web framework).

The secondary reason was that my wife was the main client, and she doesn't respond well to raw JSON. Each tweet would be just the cat's name, photo, and a link to the website. I also did some filtering as certain cats have safety requirements we couldn't meet, e.g. no neighbouring cats, no children)

One of the main issues I had to figure out early on, was "how do I distinguish which cats are new, compared to the previous response?". This was made harder because I couldn't rely on the ordering; occasionally previously-posted cats would have details updated and they would move position. Postgres UPSERT was new (to me, at least) at the time, and it seemed like a very handy way to offload the responsibility. There were never more than 50 cats listed at any one time, so it was reasonable enough to request all the animals at once, and let the database figure out which cats were new, based on a combination of identifiers that would make them unique. I could also filter the updated records to see _what_ had been updated, e.g. the cat had now been rehomed.

Another thing Elixir did really well was the polling mechanism. It's absolutely trivial to spawn a worker that can repeatedly perform a task and asynchronously hand it off to be processed.

Hope that answers your question!

I spotted that the website served its data to the frontend via an unsecured internal JSON API, so I built an Elixir app that would poll the API endpoint and upsert the cat data into the database. Any new records would get posted to a twitter account (a free way to get notifications on my phone).

It worked beautifully, and when a black cat called "Fluff" popped up, we both knew he was the right one, and we were able to phone them and arrange a meeting before anyone else. Fast forward five years, and he's sitting next to me on the sofa right now, purring away.

[0] https://github.com/ueokande/vim-vixen

Not sure what you mean by 'Phoenix on Gleam', but you could write core code in Gleam and use it in an Elixir/Phoenix app, no probs

https://dev.to/contact-stack/mixing-gleam-elixir-3fe3

All the people disagreeing with OP in this thread are misreading them as wanting to somehow ban jokes and humour.