Neverclick is a desktop application that I created when I was suffering from RSI. It uses local computer vision to detect clickable UI elements on your screen so that you can use your keyboard to perform mouse actions.

Most software in this space relies on accessibility APIs but Neverclick uses computer vision. This means that it works in every app. Additionally, it also produces really good bounding boxes around text so it's great for editing text and coding. And counterintuitively, it runs significantly faster than the traditional accessibility api approach.

It's 100% offline. It's not a wrapper around ChatGPT or Claude or any other online AI service. The computer vision system runs on your computer. I've optimized it is as best as I can, the hints should show up instantaneously even on 10 year old hardware. You don't need a powerful gpu to run it.

Also I swear on my life that it wasn't "vibe coded", I've been working on this since long before the recent AI revolution.

Comments URL: https://news.ycombinator.com/item?id=48993199

Points: 2

# Comments: 0

Neverclick is digitally signed with Microsoft Trusted Signing (now known as Azure Artifact Signing). I had to send them my address and ID to get my identity verified so that I'd get approved for it.

Is there anything else I can do to put your mind at ease?

I'm not anonymous, this is my name irl.

Neverclick is digitally signed with Microsoft Trusted Signing (now known as Azure Artifact Signing). I had to send them my personal information like my address and ID to get my identity verified so that I'd get approved for it. Additionally, Neverclick doesn't require admin privileges to run, and it works completely offline so you can block it via firewall and it'll work just fine.

Is there anything else I can say or do to put your mind at ease?

When Neverclick generates the key combinations, it prioritizes keys in the center of the keyboard. It then sorts the key combinations from most to least convenient (so for example JJ and FF would be at the top of the list). It then assigns the most convenient key combinations to UI elements closest to the center of the screen because that's where people are most likely to want to perform a mouse action.

As the other commenter already said, with dynamic data, you can't really make things repeatable. However, it's still really cozy to use and way better than if it were randomly generated and assigned (which is how it used to work before it was updated to the new system).

I've had a poor experience with accessibility apis, they're clunky, slow, and unpredictable, and as you said many apps aren't built with accessibility in mind. With computer vision you don't have to worry about that.

This might sound strange, but the cv system in Neverclick actually runs significantly faster than UI Automation (which is the accessibility api on windows) in nearly all of my personal tests. I can't believe that accessibility apis are so poorly optimized that raw pixel analysis is faster lol. Also, I have users that use Neverclick on 10 year old hardware and they tell me that the cv runs instantly whereas UI Automation is super laggy for them.

Also, accessibility apis typically don't give you useful bounding boxes for text in text editors, this is another advantage of the cv approach.

I really wish the title of the post was "Desktop application for performing mouse actions with your keyboard using computer vision" so that it's more obvious that it uses cv, although it's my fault since the current title is how I have it in the readme and they probably just copied that.