First, let me say that I’ve no problem with crawlers, whatever they might be. There’s plenty of different “security” crawlers and such that are probing servers out there, in my own case including from various technology schools and whatever – with one example being from a catholic school – but few things ticks me off quite as hard as when they do not identify themselves properly. Sure, go ahead a knock on my stuff, but if you’re legit, then you better identify yourself properly.
Now we enter the sphere of Google Cloud and AS54538 – Palo Alto Networks’ “customer presence scanner.” For whatever reason, these guys’ probes comes from two sources, one being Google Cloud hosted servers – which identify themselves as they should – but the rest doesn’t and ends up getting caught in my settings. Next, care to guess what their shitty scanner’s solution to getting a JS Challenge from Cloudflare is? Hammer the same URL for N times, each and every time. Now, that would not have be anything to really care about, had it been just your typical shitty C&C script scanning the web for exploits, but this is supposed to be probes from a “professional cybersecurity firm.”
In this case of PAN, sure I could’ve perhaps considered having their bot whitelisted, if it had behaved as it should. But nah, that’s not gonna happen since those absolute troglodytes can’t into the most basic of stuff.