Hacker News: ashish004

New comment by ashish004 in "Show HN: Finalrun – Spec-driven testing using English and vision for mobile apps"

ashish004 — Tue, 07 Apr 2026 22:59:46 +0000

Just updated README.md, it's lot simpler and addresses on the core. Thanks for the feedback, please checkout

New comment by ashish004 in "Show HN: Finalrun – Spec-driven testing using English and vision for mobile apps"

ashish004 — Tue, 07 Apr 2026 18:37:09 +0000

Yes gavinray, It gets persisted to the project. Its lives alongside the codebase. So that any test generated has the best context of what is being shipped. which makes the AI models use the best context to test any feature more accurately and consistently.

New comment by ashish004 in "Show HN: Finalrun – Spec-driven testing using English and vision for mobile apps"

ashish004 — Tue, 07 Apr 2026 17:53:17 +0000

Hi Avikaa, finalrun provides skills that you can integrate with any IDE of your choice. You can just ask the finalrun-generate-test skill to update all the test for your new feature.

New comment by ashish004 in "Finalrun – Spec driven testing using plain English and Vision for mobile apps"

ashish004 — Tue, 07 Apr 2026 15:06:34 +0000

We have built a QA agent that can understand your plain english intent and uses vision to reason and navigate the app to test your intent. You can check our benchmark here https://finalrun.app/benchmark/ and how we architected our agent for the benchmark https://github.com/final-run/finalrun-android-world-benchmar.... Its all open source

Show HN: Finalrun – Spec-driven testing using English and vision for mobile apps

ashish004 — Tue, 07 Apr 2026 14:33:48 +0000

I wanted to test mobile apps in plain English instead of relying on brittle selectors like XPath or accessibility IDs.

With a vision-based agent, that part actually works well. It can look at the screen, understand intent, and perform actions across Android and iOS.

The bigger problem showed up around how tests are defined and maintained.

When test flows are kept outside the codebase (written manually or generated from PRDs), they quickly go out of sync with the app. Keeping them updated becomes a lot of effort, and they lose reliability over time.

I then tried generating tests directly from the codebase (via MCP). That improved sync, but introduced high token usage and slower generation.

The shift for me was realizing test generation shouldn’t be a one-off step. Tests need to live alongside the codebase so they stay in sync and have more context.

I kept the execution vision-based (no brittle selectors), but moved test generation closer to the repo.

I’ve open sourced the core pieces:

1. generate tests from codebase context 2. YAML-based test flows 3. Vision-based execution across Android and iOS

Repo: https://github.com/final-run/finalrun-agent Demo: https://youtu.be/rJCw3p0PHr4

In the Demo video, you’ll see the "post-development hand-off." An AI builds a feature in an IDE, and Finalrun immediately generates and executes a vision-based test for it verifying the feature developed by AI.

Comments URL: https://news.ycombinator.com/item?id=47676044

Points: 22

# Comments: 8