The Product Idea List
A design retrospective with the best product ideas from a year of scraping people’s tweets on Twitter.
In 2020 I created a website called The Product Idea List. I was so proud of that name. It could be read as “The Product Idea List” or “The Product Idealist”. I was positive it would be a smash hit and couldn’t believe the domain was available.
As time would tell, the site did not live up to these expectations. According to Google Analytics it looks like the site has been visited about 72 times in the past year. I think I’m 40 of them.
Can’t say I’m surprised. It stopped functioning shortly after release and that’s in large part because the code is not written very well. I’m not too sour about it. Making the site was my first opportunity leveraging concepts and technologies I would have never explored otherwise and that was way more valuable than the site. I even remember writing a tweet about it at the time:
What did the site do?
It’s worth explaining that when the site was working the concept was pretty simple: Every day it would use the Twitter API to search for tweets that contained specific phrases, hashtags, or words that were related to product ideas. Phrases like “I wish someone would make” or “does anybody know of an app”.
It would gather these tweets and allow users to filter and categorize by likes or retweets to assist finding the most popular product ideas on any given timeframe.
Did it work?
Eh… not really. I would say 95% of the tweets weren’t actionable or relevant “product” ideas. It turns out three things rise to the top about 100x more than any sincere product idea does:
- Jokes: Usually ideas or intentionally outlandish concepts like:
2. Porn. My heavens I couldn’t believe how much porn would rise to the top before I filtered out certain keywords.
3. Politics. The phrase “wouldn’t it be awesome if…” was almost always related to politics.
Why the database?
I set the database up because the Twitter API limits the search to a 7 day window and rate limits the amount of calls that can be made. I was positive with the extremely high amount of traffic the site would be generating that this was a problem so I setup a cron job to search Twitter every hour and load the results to a database.
Idea statistics
Turns out setting up the database was a blessing because though the site isn’t all that great, for the past 9 months it has been steadily collecting ideas every hour. I thought it would be interesting to inspect that data for interesting patterns and see if there are any hidden gems worth sharing in a “Best Of” roundup.
Initial statistics
- Days scraping Twitter: 194
- Amount of ideas collected: 13,285
Phrases
- “amazing product idea”: 2
- “an app where”: 1,916
- “app idea”: 1,900
- “startup idea”: 449
- “does anyone know of an app”: 433
- “great app idea”: 28
- “i wish someone would make”: 1,908
- “i wish there was a service”: 357
- “i wish there was an app”: 1,775
- “new product request”: 1,364
- “startup idea”: 1,900
- “wish i could find”: 1,920
- “wouldn’t it be awesome if”: 1,900
Likes
- Tweets with less than 10 likes: 15,473
- Tweets with 10-100 likes: 320
- Tweets with more than 100 likes: 9
Retweets
- Tweets with less than 10 retweets: 12,843
- Tweets with 10–100 retweets: 350
- Tweets with more than 100 retweets: 115
My 10 Favorite Ideas
I had to search pretty hard to find 10 ideas. All-in-all I would grade the effectiveness of the website at a 2/10.
