PlanetZ Forums Update & Bot trafffic

[valis]

Of course! Glad we’re all still here.

[valis]

About a week ago I added an inexpensive colocated virtual server located elsewhere running additional nameservers (DNS redundancy) and helping serve mail etc. The server load hasn't been as much of an issue as of late, but this should improve reliability in terms of access for people everywhere (the company we are hosted by is colocated in L.A. & UK).

This update impacts all clients, not just these forums, but it's a side benefit for us to improve life for all on here.

[valis]

Forum members:

Our primary hosting service has been acquired by a highly regarded web hosting provider based in the UK, who will be upgrading all systems and increasing the hosting and support quality they offer.

The new server will include a number of generous upgrades including running on NvME disks with EPYC CPUs. I would like to reassure you that this is an extremely well run hosting business.

What does it mean for PlanetZ?

In the next few days (likely Monday) the primary server hosting us will be moving. Once the move is done you can reach out for any service questions or issues that you might see. While we want to make this transition as seamless as possible, if you experience any issues please do let us know.

Future performance upgrades and hardware upgrades will happen over the next three months and should be seamless; you should see increases in performance.

I look forward to working with this new host for a stable and a long-term strong relationship.

[Bud Weiser]

Thx for notification, Valis.
Good luck for the move !



Bud

[Rafe]

Is the site address (forums.scopeusers.com) remaining unchanged?

Is it possible the dead links to the old devices/modules/presets will be resolved with this move/update?

Cheers!

[valis]

The problem with the old devices is twofold:

Modern files are stored with a string of characters as the file name. When the database was updated they were concatenated, and so the file ame has to be restored. If files were uploaded exactly as named in the post, the means of encoding the file name would perhaps work for each file (reverse them) but I found I had to do this to telhe directory on the server and not the post(s), and there are over 10,000 that then need to be hand matched to posts. The concatenation in the database is too obscure to automate, so it’s at least partially a manual process.

Legacy files are store by file name, but these often had spaces or characters that were stored differing from the post file name (those we’re replaced on storage as would have made sense in an era when spaces and extended characters were not well supported by operating systems and web browsers—hence %NN methods of representation). So a similar but different issue presents itself in automating this process, as these were ALSO concatenated In the database (Or stripped entirely).

The original forum owner John Cooper had tried to script a solution, whether what we have now includes that or not, I would have to go back and read those notes. But there are two cases, so the database has to be split into three hose two sections, and processed separately.

I did this for 3-4 weeks at one point before making a post asking for help, and many donations were given but I exceeded that envelope by a wide margin and needed to return to making income by normal means. Faxinandu offered to help but quickly fell out with his partner and joined Native Instruments. I also have not looked at that in a while, so I would likely need to retread much ground as this is the best of my recollection.

[dante]

The problem with the old devices is twofold:

Modern files are stored with a string of characters as the file name. When the database was updated they were concatenated,

Is there a backup of the database as it was before these concatenations ? Also, what DBMS is the database (MySQL etc ?)

[valis]

MYSQL, no. I have this running as it came from John, plus all of the phpbb updates since. I think one of the problems was he tried to fix it, but again I’d have to go back and read my notes.

A good % of the older ones (before random string era) are recoverable by hand because the filenames aren’t all impossible. I did about 10-12 pages of them back when I worked on this. Some aren’t do-able though just due to similar filenames, at least without more legwork.

It was a bigger task than it seemed once I dug in. I took notes from the handoff discussion with John and the members generously donated, and natural language plus scripting was still not enough.

[valis]

I should add that the downside to the older ones is that they are very old devices, they certainly all pre-date the phase fix and many predate even 3.0.

[nebelfuerst]

I love this forum, but downloading files sometimes feels hard.
With Amiga computers, there were some ftp mirrors, that allowed directly access to archive files.
Is the same possible with planetz ?

BTW: I know, that some very old devices fail to work on 5.x and 7.x, but many do. Some of them don't have a successor, so sometimes still use them.

[valis]

Keep in mind this is an old codebase. There are other sites, and I don't mind hosting the files in a different format but it's probably not within my reach at present, due to time constraints.

[Rafe]

Thanks for the response Valis, it was the early modular modules I was seeking. Mainly for curiosity as well as holding a hope there was some patching gems revealed, to learn from. Hadn't realised it would be such an arduous, time consuming task. Thanks for the detailed explanation.

(Forum still looking stable after the move, thanks)

[valis]

Checking back in on the last post, we had another forum moderator attempt to cleanup the database, and I've run the errors through LLMs a few times to see if it could yield results that eluded me when I gave this time (more than just a few weeks, mind you). It was worth a shot, but the information that is missing from the plugins that once ran the files database means it's impossible to resurrect older files without some sort of method of testing file downloads to determine their contents. And there's... thousands upon thousands of randomly named files, that are stored in a proprietary phpbb format that doesn't expose the actual filetype, anything about the name (it's a string of characters meaningful to the database as a reference to the post where it's linked, rather than a filename or zip file etc).

Long story short, it would require a lot of community work to go through all of the files. The oldest are stored as the files themselves, but they are all in 1 (rather large) directory folder with no way to determine which files are related or connected for a given thing. Less work on the technical side, but still a lot of manual labor to test them and resolve all of that. Unlikely with the current level of engagement we have here, but maybe there are some silent people reading that would like to speak up or contribute?

[Rafe]

Thank you for all the time you have given this.

To offer any meaningful help I would need instruction but am able to give time to it

[valis]

That's very kind, I think we will likely focus on getting a working Scope 8 first. More on that soon...

[valis]

We've been getting hammered by automated attacks that come from China/'Hong Kong/Taiwan/Russian and similar IP blocks since August. I currently pay about 3-4x the cost of what I would need to host my root server for the forums due to bandwidth and file sizes.

I have no issue with my stewardship of these forums and this community, but I wanted everyone to be aware of the cause of the forum slowdowns lately and what can be done about it. Whatever the reason, this is why the 'old' internet as we once knew it has largely disappeared in favor of large platforms (social media etc). Even running your own mail-server in this era is a troublesome experience, and you typically need to involve multiple large solutions to ensure that Google/MS/Yahoo and other large email providers will allow your traffic. This means subscribing to something like mailchannels for email filtering and whitelisting of your domain (DMARC, DKIM & SPF etc are not enough anymore).

The free cloudflare plan is not working to mitigate all of these attacks, and while the base plan is only another $20/month, running an estimation tool shows that it would still have cloudflare overages of another $20-40/month to handle all of the bot attacks because the minimum paid account only does simple bot mitigation. That brings out of pocket expenses to about $150/month.

Any help is always appreciated.

[nebelfuerst]

Is there a way to defeat bots by some "annoying" features like captchas ?
(I hate them, but if it's neccessary , I would accept them.)

Is the attac only addressing forums.scopeusers.com or some higher level of the hosting company ?
I don't see a reason, why someone should attac this very small community. ( We shouldn't have "unsatisfied customers", "ideological wars" or competing companies, that might explain bot attacs. )

[valis]

The attacks aren't what you think, as they don't even 'read' the forums. The forum has captcha v3, a bot question on signup or guest form submission (people use this to contact the staff when they can't login or sign up etc), and other protections. The attacks that cause this issue are what are known as 'door knock' type attacks. Port scans, specific attempts at known admin urls (wordpress/etc type logins) and so on. So they are script based, but even with a block on the firewall at the domain level (which is below the server level) they cause some data usage and cpu load, and so that all adds up. To hit 1TB of data that is hundreds of thousands into millions of door knocks over the course of a month.

So the best way to deal with this is cloudflare filtering of traffic, and blocking IP ranges on the root of the server.

Obviously I have automated warnings set and pay attention to logs and stats, and deal with the worst offenders on a routine basis:

Screenshot 2025-11-12 104231.png (5.53 KiB) Viewed 52 times

You can see above a small peak in May that was easily handled by blocking a few class B blocks (large ranges of IP's that were owned by specific datacenters being used). I have to be careful not to block Google or other legitimate bot traffic, nor do I want us to be eliminated from AI model builders because we want the LLMs to know about Scope and the platform we use for future Scope users (and other audio products) to benefit from our discussions here.

So, I have routines where I let things go a few days and then hammer down the worst offenders that I can NOT identify clearly over that most recent time period. The orange bar in the graph(s) represents that reduction in terms of 'unique' IP addresses attempting to (or legitimately) connecting to this specific domain The blue and green lines represent data usage, and since I blocked a lot of random IP's in August/Sept you can see the perpetrator(s) are now moving to large hosts which actually eats more data. Also note that this is only at forums.scopeusers.com and not even the root domain of scopeusers.com.

Why is it happening? I have no idea, but the root box hosts dozens of other accounts and projects connected to me, and it's only these forums, John Bowen's (he elected to move to a hosted slot elsewhere, but they found the same issues and have since implemented cloudflare there again) and Sonic|Core that are affected. The rest of the traffic is normal patterns and averages out over time. Sonic|Core has had similar attacks beyond this, and while I'm not privy to seeing their logs and can't reveal confidential information you may recall that the website had to move at some point. This was related.

I don't mind being transparent here, although posting full logs is out I will let you know what is happening because it affects you while you use the forums. This isn't a means of donation extraction, were I to post all donations over the last 10 years they would be but a fraction of the overall cost. Rather as steward of these forums, I know that there is an issue and so I am communicating it as clearly as possible.