New ask Hacker News story: Hypothesis: Repeating the task description increases quality of ChatGPT output

Hypothesis: Repeating the task description increases quality of ChatGPT output
3 by kuboble | 2 comments on Hacker News.
There has been some experiments showing that ChatGPT performs better if given some incentives like tips or threats, etc. Also it's known that the chat GPT performs constant amount of computation per token. I wanted to test a hypothesis that adding any number of tokens after the initial task description increases quality of the output. The experiment consists of relatively simple coding tasks and we will compare two prompts: Please help me X. and I will provide an identical task description 10 times: Please help me X. Please help me X. Please help me X. Please help me X. Please help me X. Please help me X. Please help me X. Please help me X. Please help me X. Please help me X. I have decided to run 3 experiments and not cherry-pick the results. Experiments: 1) create an SVG element of a 5 edged star item 2) write a function to check if a number is prime in python. 3) write a function that given chess position in FEN notation as an argument returns which side has material advantage in python. On the task 2) both prompts returned exactly the same correct answer. Results for 1) https://ift.tt/DSQRM8X Results for 3) https://ift.tt/e4l7JrD For the task 1) clearly and for the task 2) arguably the results are in line with hypothesis that simply increasing prompt length leads to better results. Does anyone have similar experiences / can check that with other short coding prompts?

Comments

Post a Comment

Popular posts from this blog

New ask Hacker News story: Tell HN: Equifax free credit report dark patterns

Tell HN: Equifax free credit report dark patterns 4 by PopAlongKid | 0 comments on Hacker News. For years, I have been obtaining free annual credit reports (annualcreditreport.com) which must be provided by law. Recently, for the first time, when I tried to obtain my Equifax report, I was prompted for an email address and a mobile phone number, a new requirement that apparently cannot be bypassed. The other two bureaus, and Equifax previously, confirmed identity by asking knowledge based questions. They do not need my phone or email for anything. Next, trying to obtain the report by phone instead of web site per instructions, I got to the point where I entered my ZIP code on the phone keypad, the voice menu system correctly repeated the number back to me, but every time I press 1 to indicate it is correct, the system acts like it got an invalid response and only gives me the option to enter further information by voice, not by the phone keypad. Just as with my phone number and email...
Read more

New ask Hacker News story: Ask HN: Why can't the US government run their own social media?

Ask HN: Why can't the US government run their own social media? 2 by rglullis | 2 comments on Hacker News. With the news yesterday of POTUS joining threads.net and with the fanfare about the possibility of following him from other AP-compatible servers (like Mastodon), I was left wondering why is it that the White House couldn't just run their own services? Doesn't "POTUS is using threads for social media" feel a bit like "POTUS is using outlook.com for email"? Is there anything about the bureaucracy that stops their some small team to setup a Mastodon server on "social.whitehouse.gov" and make this a reality? I find it hard to believe that this would be "if it got hacked it could create diplomacy issues", because I don't think that Facebook is setting up any type of special infrastructure to deal with political figures on their platform. Also, none of this seems to be a problem for some of the EU governments. Maybe I am missin...
Read more