METR measures time horizon but notes its limitation in how it might be misleading (limited in scope, ignores tacit knowledge and context of the job). Epoch AI measures and predicts computation, but it is merely a proxy for what we actually care about. Anthropic had to rely on internal metrics because metrics saturate. The two problems underneath, measuring it and defining it, turn out to have the same shape, and so does the way out.
This measurement problem leads to the demand for more rigorousness in the AI safety community, a standardised way of how AI safety can be tackled. One approach may be Bayesian safety? I need to read some articles about it to discuss here.
However, rigour and theory may not always be the best. Take the trajectory of DL vs RL, RL is more theory-oriented and proves everything mathematically, but could not beat the quickness and engineering of DL. Eventually DL was given theory, but even now, they are results-first, in contrast to the theory-first RL. We need AI safety measures quickly, and we cannot battle with all-theory rigour against the fast, ship-first labs racing to scale.
This can be resolved by taking the middleground: Have one core theory that we can stand on, and use that as the basis, not the tools that must be applied all the way up the stack. Ultimately, RL was used as tools for RLHF and RLVR. We may be able to do a similar thing. For Bayesian safety, instead of insisting on Bayesian methods, take the concept of measuring alignment with Bayesian beliefs from theory, and then apply any other method that will improve that.
There is also an undermining problem of definition. What is intelligence? Can an individual be more intelligent than another? For example, for any given task, one individual will perform better than the other? Maybe not for humans, since humans depend on the environment. Take business, for example. It involves so many aspects including strategies, vision, connections, etc. Is it possible to measure them all in a single aspect?
This can also be resolved by taking the middleground: Have one fixed core theory that we can stand on, and empirically iterate and improve the theory on top of it. We can definitely agree that something is happening so we must address the issue. So we don’t need to theorise from first principles in order to work effectively. We can assume some definitions, work on that and see its limitations, and then come back to refine the definition itself.
Also, verification-easier-than-generation is a plausible, or the only exploitation we can make use of in AI safety. For data scaling, because verification is easier than generation, we can improve AI through synthetic data generation and distillation. Similarly, if we improve upon the definition and method of AI alignment verification, since checking is cheaper than producing alignment, this asymmetry will allow verification to be proportionally improved to how much we specified the target.