Ernie.QT
patreon
Common Alpha 009: DSR - Alternative Metrics
🕑 Added 2024-12-27 16:17:18 +0000 UTCThis content is to introduce a new metrics that has been proposed in financial engineering world in this recent years. It is the ‘Deflated Sharpe Ratio (DSR)’, that actually helps to reduce a lot of blind spots of only using Sharpe Ratio.
Probabilistic Sharpe Ratio (PSR)
Before talking about DSR, here I will use a simple example to explain why purely Sharpe Ratio itself is not a good enough metrics to evaluate strategies & introduce a new term ‘Probabilistic Sharpe Ratio (PSR)’. When we use sharpe ratio itself to evaluate strategies, although sharpe ratio itself is easy to understand and quantify, we failed to incorporate the 3rd & 4th moment of the return distribution. And PSR attempts to add the confidence interval concept into quantifying sharpe ratio.
Example:
Strategy X
Long term true weekly mean returns (Strategy X): 0.20%
Long term true std returns (Strategy X): 0.60%
Long term true skew returns (Strategy X): -0.99
Long term true kurt returns (Strategy X): 3.86
Sharpe Ratio: 1.62
Strategy Y
Long term true weekly mean returns (Strategy Y): 0.32%
Long term true std returns (Strategy Y): 0.60%
Long term true skew returns (Strategy Y): 0.00
Long term true kurt returns (Strategy Y): 3.00
Sharpe Ratio: 1.55
If we try to evaluate only on Sharpe Ratio, we will choose Strategy X since it has a higher sharpe ratio. However if we try to evaluate on PSR; it indicates that despite of the lower SR of the second strategy, this strategy Y have more chances that in the future its returns will have a TRUE SR greater than 0; and we can say with a 95.19% confidence that the strategy Y it's a winner strategy, but we "only" have a certainty of 92.98% that the strategy X can generate alpha too.
The Probabilistic Sharpe Ratio is a powerful statistic that gives us the confidence level associated with a particular SR estimation.
The estimation errors on the SR increases with negatively skewed returns and fatter tails (big kurtosis). In those cases, it will be needed a longer track-record for decreasing these errors.
Deflated Sharpe Ratio (DSR)
The Deflated Sharpe Ratio (DSR) is designed to correct for two main sources of performance inflation in financial strategies: selection bias under multiple testing and non-Normally distributed returns. Here are its key features:
Selection Bias Correction
Selection bias occurs when researchers or analysts report only the positive outcomes of their tests, ignoring the negative ones. This leads to an overestimation of the strategy's performance. The Deflated Sharpe Ratio (DSR) corrects for this by considering the number of independent trials conducted. By adjusting for the number of trials, DSR provides a more accurate measure of the strategy's true performance. This helps to avoid the "winner's curse," where the chosen strategy appears better than it actually is due to the sheer number of trials conducted.
Non-Normality Adjustment
Financial returns often do not follow a Normal distribution; they can be skewed or have heavy tails (kurtosis). The traditional Sharpe ratio assumes Normality, which can lead to performance inflation if the returns are not Normally distributed. DSR adjusts for skewness and kurtosis, providing a more accurate measure of performance by taking into account the actual distribution of returns.
Multiple Testing Adjustment
When multiple strategies are tested, the probability of finding at least one strategy that appears to perform well by chance increases. This is known as the multiple testing problem. DSR adjusts for this by considering the number of independent trials and the variance of the Sharpe ratios tested. This adjustment helps to ensure that the reported performance is not just a result of random chance.
Probabilistic Sharpe Ratio (PSR)
The Probabilistic Sharpe Ratio (PSR) is a component of DSR that computes the probability that the true Sharpe ratio is above a given threshold. PSR takes into account the sample length and the first four moments of the returns distribution (mean, variance, skewness, and kurtosis). By doing so, it provides a more robust measure of performance that accounts for the sample size and the distribution of returns.
Formula:
The Deflated Sharpe Ratio depends on
the estimated Sharpe ratio SR^,
the sample size T, e.g. on how many trading days T the strategy has been backtested,
the first four moments of the alpha / strategy / manager / fund returns to discount for exceptional returns.
DSR is a useful concept when it comes to some strategies generated by Machine Learning method or any methodology that involves the risk of data snooping as to avoid overfitting and biases. The below example shows us how easy we can actually fooled by sharpe ratio and that is why sometimes after you find a strategy, the whole next year can be unprofitable in production. The formulas and codes are all attached in the zip files.
