[plug] Hiding in plain sight

[Brad Campbell]

G'day All,

We run a Zimbra server and have done since 2011. Zimbra is such a tangled ball of wax held together with string and chewing gum, ongoing vulnerabilities are inevitable. It's also reached a critical mass where exploits are pretty prevalent. Plus they've dumped all staff that knew what they were doing and offshored all code development. It seems to be in a tail spin, but that's another story.

Back in 2022, there was an really serious exploit going around that took Zimbra "a while" to mitigate. The quickest mitigation was to put a nginx reverse proxy between Zimbra and the world and use it to close that hole. That seemed like a pretty good idea anyway, so I worked one up. We host about 4 domains on the server, and there is zero public information about these domains. The certificate for both Zimbra and the proxy requires the various domain information, so I set up a dummy domain as the CN of the certificate, then placed each domain in the cert as an additional entry with a wildcard like *.frobble.com. 

This worked _really_ well. Both Zimbra and the proxy were set up to 418 any request that wasn't specifically for a required domain, or in the case of a specific domain, a specific location and the scans dropped off within a week. Since then, we log about one lazy scan a fortnight whereas previously it was several 1000 a day.

Last week, I did a review of our certificates and thought I should really tighten things up. No point holding a wildcard when all I needed for that particular domain was autodiscover.frobble.com. *big* mistake.
I didn't realise that cert authorities had to publish transparency logs, and the bots all scan these Within 10 minutes of me issuing the new certificates the servers started getting *hammered* with scans. Suddenly all my previously opaque subdomains were exposed for all to see. From 1 scan a fortnight back to 1000's a day.

On the bright side, I had left the actual zimbra server domain wildcarded, and the autodiscover domains all regex filter the proxy location, returning a 418 for everything else.

So, I've reverted the changes to the certificates and I'll wait for it to all die down again.

I've made a *big* note in my changelogs as to why I've reverted back to wildcards, and in my best Michael Caine cockney voice "Don't do it again".

If you publish a certificate (and lets face it with LetsEncrypt these days everyone should), consider what you want to be made public when putting it together.

Regards,
Brad