Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Workaround for slow search index refreshing & more #584

Open
wants to merge 13 commits into
base: master
Choose a base branch
from

Conversation

ImNotIcon
Copy link

@ImNotIcon ImNotIcon commented Jan 21, 2024

I want to preface this by saying that I've deleted over 200k messages with these changes in the span of ~7 days and my account hasn't gotten banned. The fastest I've been able to delete messages without any problems was a little less than 1k per 30 minutes.

This PR has the following changes:

  • Workaround for Discord's slow search index refreshing: Discord takes ~35s to reload its search pages, we work around this by alternating between ascending and descending sort order on each search.
  • Fixed prematurely ending the job when the API returns an empty page: added checks to make sure if this is the case, if not we just wait until Discord updates the index.
  • Fixed spamming the log when search gets rate limited & disabled increasing the delays: after getting rate limited once, the delete delay would change to whatever Discord's retry_after was and then it'd be a cycle of getting rate limited and changing the delay again.
  • Added rate limit prevention: through trial and error I've found that 45 requests per 60s and 4 requests per 5s are Discord's rate limits for users, this however might need some testing. I've barely had Discord rate limit me with that change while also keeping the program's message deleting speed relatively fast. There is an on/off checkbox for this option.
  • Search errors will just return no messages instead of crashing: the job would crash, defeating the purpose of "wipe archive" since it wouldn't be automated.
  • Added a button to include servers when using the "wipe archive" option, however "wipe archive" doesn't work with servers yet.
  • Workaround for the program getting throttled when being on another tab: browsers make timers run slower to preserve battery and memory, which would make deleting messages slower.
  • Added an option for trimming the log: when I had this run overnight, the tab would crash from the log text being too long. I've found that at ~500 lines in the log is when things would start getting laggy.
  • Fixed estimated time remaning calculation: it now factors in your ping and the messages deleted so far.
  • Wipe archive now supports server channels using the Discord API to fetch the guild id. There is an on/off checkbox for whether to include server channels or not.

I've also changed search delay and delete delay to 1400ms. Through testing, this was the fastest I could go without getting rate limited all the time.

Works around Discord taking too long to reload the search page. It sorts messages from start to end while the end to start page is loading so there is no time wasted waiting for pages to load.
Browsers throttle the JS timers when on another tab, therefore leading to delays being longer than they are supposed to
For larger operations with a lot of messages, the tab starts lagging after a couple thousand are deleted
@ImNotIcon
Copy link
Author

Conclusion: I deleted 650k messages in total with these changes without any interruptions, it took me around 17 days of my computer running 24/7. My account did not get banned in the end.

@Aida-Enna
Copy link

This sounds amazing - How would we modify the current script to use the updated files? I assume you could just copy the main JS file, but I'm not sure about the UI files, I never use tampermonkey except for this.

@Dalethium
Copy link

Dalethium commented Mar 2, 2024

This sounds amazing - How would we modify the current script to use the updated files? I assume you could just copy the main JS file, but I'm not sure about the UI files, I never use tampermonkey except for this.

@Aida-Enna
View contributing.md, the long and the short of it is you need to build it using npm. I would post my copy here, but, I should note that you should never run random code somebody posts unless you understand what it does -- even if you trust them, especially in this context. I would strongly recommend learning to build it yourself, you will need a device with nodejs on it, then you open the root folder of the program in command and run npm i, then npm start -- then visit the link it sends. I will note that the changes made in this PR don't look malicious to my eyes, and I am personally using them myself and they seem fine, but again, you really should not run code just because someone says it's ok, especially on projects like this.

@Aida-Enna
Copy link

This sounds amazing - How would we modify the current script to use the updated files? I assume you could just copy the main JS file, but I'm not sure about the UI files, I never use tampermonkey except for this.

@Aida-Enna View contributing.md, the long and the short of it is you need to build it using npm. I would post my copy here, but, I should note that you should never run random code somebody posts unless you understand what it does -- even if you trust them, especially in this context. I would strongly recommend learning to build it yourself, you will need a device with nodejs on it, then you open the root folder of the program in command and run npm i, then npm start -- then visit the link it sends. I will note that the changes made in this PR don't look malicious to my eyes, and I am personally using them myself and they seem fine, but again, you really should not run code just because someone says it's ok, especially on projects like this.

Ah, thank you. I had no idea build instructions could sometimes be in the contribution readme. And yeah, I looked over the changes and they looked good, I just didn't know how to put it all in one file. Thank you!

@ImNotIcon
Copy link
Author

For my personal testing, I uninstalled my current installation and created a new script on Tampermonkey with the code from the deleteDiscordMessages.user.js file. You will need to remove @downloadurl from line 14 as it will auto update and revert the changes.

@ghost
Copy link

ghost commented Mar 15, 2024

I tried this fork and deleted more than 150k messages 24/7 without a ban. The only setting I changed from default is delete delay to 1500 instead of 1400 because it significantly reduces the number of rate limits.

As a note:

  • I only used my account for calling while the delete script was running. This means no reading messages and no sending messages.
  • I stopped the script every time I switched away from the channel the script was running on

@thecatontheceiling
Copy link

Does this still work fine? Used it on my account and after 4 hours it was still going with very minor ratelimiting

@ImNotIcon
Copy link
Author

It should still work perfectly as far as I'm aware, I was told setting both of the delays to 1500ms would be better to avoid rate limiting

@thecatontheceiling
Copy link

I got ratelimited 90 times in 6 hours using default settings lol, it said total wasted time was only 49 seconds though

kazuki388 added a commit to kazuki388/Undiscord that referenced this pull request Nov 15, 2024
- Resolved ESLint issues and addressed invalid header warnings (victornpb#663).
- Implemented autofill for thread channel IDs and restricted message deletion to the corresponding thread (victornpb#610).
- Clarified that omitting a time will disable the date filter functionality (victornpb#643).
- Added a filter to exclude messages containing files or attachments (victornpb#642).
- Fixed various autofill issues (victornpb#631).
- Stopped execution when there are remaining messages (victornpb#619).
- Addressed API errors related to the Wipe Archive Feature (50024 - 50021) (victornpb#603).
- Implemented a workaround for slow search index refreshing (victornpb#584).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants