🗂️ I have Been Saving Posts from the Forum for 3 Years now. What Have You Saved… and Why?🗂️
(media.greatawakening.win)
You're viewing a single comment thread. View all comments, or full comment thread.
Comments (87)
sorted by:
I tried to do that at the start, i really did, then i got to just dumping it into one folder. Now i just use a search engine.
I thought of writing a scraper to scrape every post and comment on GA.W, and then saving it in an MS access file so any anons with MS Access could use it to search anything. But with so many users it would get pretty intense..
Scraping every username not found for all their comments and posts. Then searching all their posts for any new usernames and tossing them on to the stack to get scraped. But then it would have to be constantly constantly checking all the usernames and posts (including edit updates), at an interval for any new comments to update the database with. It would make a great project, but it would be abuse on the communities.win servers to get that initial scrape, and keep it running to check for any user edits, so i haven't done it. But i still think of doing it. It would be much better with API access to communities.win