Option to make tests fail on logging errors

[nickrobinson251]

By default the Base/Logging macros do not throw an exception if there's an error generating the message, it just logs an error-level message
e.g.

julia> @info x
┌ Error: Exception while generating log record in module Main at REPL[2]:1
│   exception =
│    UndefVarError: `x` not defined
│    Stacktrace:
│      [1] backtrace()
│        @ Base ./error.jl:114
...   ...
│     [17] _start()
│        @ Base ./client.jl:522
└ @ Main REPL[2]:1

But, whilst this might be desirable behaviour in production, this is not so desirable in tests.
You may want logging failures to throw in tests, so that logging issues can be found in tests (rather than happening in production, and subsequently having unhelpful error-level logs where you expected some other helpful logs).

runtests could provide some option like catch_log_exceptions::Bool=true which when set the false would instead throw on logging issues. And could be set via an environment variable, like ENV["RETESTITEMS_CATCH_LOG_EXCEPTIONS"] = false, or some such.

The question is how could we do this?

Unfortunately, Base / Logging does not provide an easy was to set this behaviour (see JuliaLang/julia#52143)...

We could set the logger to TestLogger(catch_exceptions=fales), which sets the method Logging.catch_exceptions(::TestLogger) = false, but running tests with a different logger than would be used in production might also not be desirable, as we'd no longer catch issues caused only by whatever logger is used in production.

We could sets Logging.catch_exceptions(::Any) = false for all tests, but this is type piracy (the default is true). And would not work is a custom logger had declared there own Logging.catch_exceptions(::CustomerLogger) = true