Historically, as a PM, you couldn't do anything without an engineering counterpart.

Now, with GenAI tools, new PMs can easily create and deploy their own apps. In doing so, you'll learn the core PM job skills (for example, thinking about user value, defining requirements, creating designs) and get introduced to the core technical work (for example, writing and testing code, deployment).

The latter is very important for PMs to understand and it's now much more accessible than in the past. You can use Cursor and various Figma to Code platforms for code, and ChatGPT and Claude to help with deployment.

When I'm looking to hire PMs, those who have actually built stuff have a leg up over other candidates.

Hope this helps!

- Evals are very useful and a core part of the best practice for creating LLM apps

- There are already excellent solutions for model and prompt eval (etc.) including Parea from YC and many others

The main reason is that the APIs represent an excellent cost / performance / complexity tradeoff.

Every project has relied primarily on the big models because the small models just aren't as capable in a business context.

We have found that Gpt4o is very fast, when that's necessary (often it's not), and it's also very cheap (gpt4o batch is ~96% cheaper than the original GPT4). And where cost is a concern and reasoning doesn't need to be as good as possible, gpt4o mini has been excellent too.

There are already many prompt/LLM routers available.

We've never found value in them, for similar reasons as mentioned above.

Notably, this comes after the release of

1) "Custom instructions" in the settings to manually modify your system prompts.

2) Retrieval as part of Assistants API (RAG)

3) The "GPT" builder allowing you to define how the system responds via chats (I assume this uses chats to define system prompts)

Feels like a natural extension of these to me. If someone extracted the new system prompt, I would expect to see a section on this similar to custom instructions.

1. Compute (H100s are almost impossible to get ahold of, especially in the quantities they'd need to train a competitive model)

2. Data (OpenAI has a very rich set of user data from ChatGPT that is necessary for creating a model like GPT-4; a new venture would lack this data -- the cold start problem -- but Microsoft may* have similarly usable data from non-enterprise Bing Chat)

* Someone please feel free to prove me wrong on this - I'm not aware of a MS policy saying they won't use Bing Chat prompts for training, but it could very well exist.

For another definition that incorporates these elements (explicitly or implicitly), see Llamaindex's high level concepts page [1]:

"Agents: An agent is an automated decision maker powered by an LLM that interacts with the world via a set of tools. Agents can take an arbitrary number of steps to complete a given task, dynamically deciding on the best course of action rather than following pre-determined steps. This gives it additional flexibility to tackle more complex tasks."

[1] https://docs.llamaindex.ai/en/stable/getting_started/concept...

1. Have a goal (hard-coded, taken as input, etc)

2. Create a plan (of steps or tasks) to achieve the goal

3. Execute (the steps or tasks in) the plan to achieve the goal (often including tool use)

4. Sometimes they check for success against the goal and/or re-plan based on current status

Love your blog BTW

Since ChatGPT was released, I've always wanted to be able to input queries by speaking instead of typing.

It just seems like it would be a much more natural way to interact with the system.

So here's a simple python script that leverages the ChatGPT and Whisper APIs to provide an experience similar to the ChatGPT web interface, but with audio input instead of text.

It works like this:

User: [Recording. Press the enter button to stop.] Hello?

Assistant: Hello! How can I assist you today?

User: [Recording. Press the enter button to stop.] How are you today?

Assistant: As an AI language model, I don't have feelings or emotions. But I'm always ready to assist you with any questions you may have!

Comments URL: https://news.ycombinator.com/item?id=34992930

Points: 4

# Comments: 1