You Need To Understand The Role of Luck and Skill

In part 6 of my 7-part series on Lessons I’ve Learned, we will look at luck and skill. In order to be successful taking risk in the markets, you must understand the role of luck and skill. From that understanding, there will be several implications for your actions and how you invest and trade.

Markets are fascinating, complex, multi-part puzzles that are constantly moving. Anything can happen. Surprises are frequent. That means that randomness and luck are omnipresent forces in the markets. I dealt with this in another post, Better Lucky Than Good. As anything can happen, we can’t put ourselves in a position where we are betting our entire account on one position.  For instance, think of an event like the Flash Crash in 2010—an extreme example, but one day like that could wipe you out. Furthermore, because luck is a constant factor, we need to adjust our own thinking and practice about investing to take luck into account.

One basic implication is that we won’t ever be right 100% of the time. Lots of things can go wrong with positions for numerous reasons, no matter how much research and preparation we have put in. As such, we should design our trading and investing strategies to reflect the fact that we can get it wrong sometimes. That means learning to manage the downside when have a losing position, which will probably happen a lot. In fact, most professional portfolio managers and traders have a win rate of around 50% over time, so they will often have to deal with losers.

Once we accept this fact, then we look at the markets differently. We learn that instead of trying to make money on every individual position, we need to have an approach that accepts the fact that we could lose money on any individual position. As I emphasized in Lesson 1, this means trying to make the right decisions instead of having a narrow focus on making money. The latter is the outcome and because there is an element of chance in the markets, we can never really control or determine the outcome. Rather, what we can control is the process itself, and that is where we should direct our efforts. Moreover, if we get the process right and never leave ourselves too exposed to a single event undermining us, then the outcomes will work out favorably. By focusing on our process, we are cultivating skill.

The most common way of evaluating trades is on whether or not they made money. But this doesn’t tell us if we made money because we got lucky or because it was the right decision. This is just focusing on the outcome. When describing the results of positions, we should break them down into either winning trades (i.e. they made money) or losing trades (i.e. they lost money).  This classification system just tells us what happened. But we want to focus on the process, because that’s what we can control. Instead, I would sort them into good trades (i.e. ones that are consistent with your process) or bad trades (i.e. ones that are not consistent with your process). No matter what the outcome on any one position, you want to be putting on good trades which are disciplined and consistent with your process. The idea is that the results will take care of themselves over time.

This all sounds great, but how do we determine if something is a good trade or just a winning trade? How do we actually know if we are being skillful as opposed to just being lucky? To answer that, we will use the language of probability and statistics. As I discuss extensively in my book, understanding probability is key to success as a trader or investor. Revisiting Lesson 1, the most basic concept is that of statistical expectation:

Statistical Expectation = (Winning % x Profit Per Winner) – (Losing % x Loss Per Loser)

For instance, if we estimate that a position has a 50/50 chance of being a winner, then the profit per winner just needs to be a bit higher than the loss per loser for you to make money long term. As your profit per winner grows relative to your loss per loser, then your required win rate goes down. If you make $6 on every winner and lose $1 on every loser, then you can be right only 25% of the time and still make a fortune. If a position has a positive statistical expectation, then the risk/reward makes sense—and putting it on is a good decision.

Now, that does not mean that every single position will make money. Rather, it means that over a sufficiently large sample size, i.e. a large enough number of positions, you will make money. As long as you consistently have favorable odds, then they will assert themselves over time.

Of course, you don’t always know the odds in advance. You can figure them out in several different ways. The first is through experience. If you’ve been using the same investing or trading strategy for twenty years, then you should have a good idea. You can draw on your records and analysis to determine the key variables, like your win rate or how much you make per winner.  The next way is historical backtesting or research, where you have looked through data and seen how your strategy would have done in the past. This isn’t the same as actually using real money, but at least it gives you some idea. The third method is to construct a new methodology and to make educated guesses about what kind of returns it may deliver. This is the riskiest because it doesn’t have as much as grounding in experience or reality, but it could work if it’s difficult to backtest or because market conditions have shifted rapidly over time. While you may be guessing, you still need to construct your strategy’s basic components in a way that leaves you positive statistical expectation.

As we have stated, you need positive statistical expectation to be profitable. But just how profitable you are will be determined by a few key variables. The first is win rate—how often a position is a winner or a loser. The second is how much you are making per winner. The third is how much you are losing per loser. Understanding these three variables, you can see how different styles work. In baseball parlance, some people try to hit “home runs”, focusing on big wins but not necessarily needing high win rate. Others try to hit base hits, focusing on a smaller average win size but trying to have a higher win rate. All of these can be successful, because they have one common attribute: positive statistical expectation.

Moreover, we can boost our overall expectation. Here is a simple diagram that shows how a couple of tweaks can dramatically change our results. And it is possible to make these tweaks. For instance, we could change our take profit policies on winners so that we hold them longer and make slightly more. That would boost our average winner and substantially boost our overall expectation. Similarly, we could boost our total win rate by cutting out one or two strategies or setups which are dragging down our overall statistics. Even small changes can have a meaningful impact on our overall statistical expectation and thus on our overall profitability.

As I discussed in Lesson 5, reviewing and tweaking our own performance is an essential ingredient for getting better. One of the most important elements is to track your own results and trading statistics, paying attention to these exact component of our results. One important thing to keep in mind is sample size. Your results only become relevant once you have a certain amount of data. This threshold can vary by style. If you are an intraday trader, you can’t evaluate your overall skill level based on a single month. A long-term investor would need several years before he could confidently evaluate his own results and contemplate changes. As all athletes know, we can get into slumps. But slumps can happen with greater frequency than we realize. For instance, if our average win rate is 50%, then we have around a 1% chance of having 6 losing positions in a row. If you only have 8 or 10 trades under your belt, then after a string of 6 losses you could incorrectly assume that your strategy has no chance of making money. Instead, you should build up your sample to 100 positions.

 This brings up an interesting tension: how to balance the effects of luck versus statistical expectation. In the long term, we want to be making the right decisions and taking positions with positive statistical expectation. We know that it will work out over the long term, in spite of the short-term influence of luck. But on the other hand, we know that things can go wrong in the short run. If we have six losing trades in a row, we don’t want to suffer a crippling drawdown. We want to be protected from ruin in case there’s another Flash Crash.

A good guideline for how much to risk is the Kelly Formula, developed decades ago to help gamblers figure out how much to wager on a bet. It is a good starting point for traders and investors, albeit with some caveats that I will point out.

f* = [p (b +1 ) -1 ]/b

f*= fraction of bankroll to wager

b = payoff to a winning bet

p= probability of winning

The math is simple. If a successful $1 wager would pay $3 in winnings and has a 50% chance of working, then we would bet 1/3 of our overall bankroll. We would bet only 10% of our bankroll if it was $2 in winnings and a 40% chance of working. Obviously, these are just suggestions. The Kelly Formula only gives us a framework for thinking about risk. As discussed previously, we won’t know the odds in advance, so it’s harder to judge the potential payoff from a winner. Some famous traders, like Ed Thorp of Beat the Dealer and Beat the Market fame, advocate strict adherence to the Kelly Formula, but it’s just a guideline.

There is one other problem: luck on any shorter timeframe. The Kelly Formula is designed to maximize long-term profits, so it will advise risking sums that will seem too large to most people. They often can’t or don’t want to stomach the volatility and temporary drawdowns associated with such large risk and therefore will want to risk less. Indeed, some of the academic studies mentioned in the book Fortune’s Formula advocate reducing the suggested risk by one half to get the right balance between results and volatility.

The Kelly Formula discussion brings up another point: the psychological impact of luck on your trading. One of the reasons to reduce risk below what’s indicated by the Kelly Formula is to make the intermittent drawdowns and the volatility bearable. Similarly, if you endured a long losing or winning streak, you would be affected by it—you would probably lose your nerve or think that you can do no wrong. While we would like to get a state where we are not impacted emotionally by our results, that is not possible. Instead we should expect that our results will impact us, especially if they are adverse results like a string of losses—and then make a plan so that we can minimize the damage.  What we want to do here is to create a certain plan that calls for cutting down risk even further if we suffer certain stumbles. By keeping risk low generally and reducing it even more when we have stretches that could unbalance us, we are ultimately doing ourselves a favor—giving our P&L and our state of mind the chance to fight another day.

Focusing on the process is the other way to minimize the psychological impact of short-term luck. As I have emphasized in a post in Lesson 1: Trading is Decision Making or my post How to Be More Consistent in Your Trading, the path to sustained improvement is to reframe trading as making good risk/reward decisions, and not as making money. Concentrate on boosting the quality of your decisions and you will be process-focused. By doing this, you are essentially tricking yourself, so that you remove the emotionality that can come with focusing on money or thinking about it too much. Rather, get into the habit of focusing on the quality of the decisions that you are making, knowing that the results will take care of themselves. This advice echoes Michael Mauboussin’s famous comment, “Where there is an element of luck, focus on the process”.

We would like to believe that we will never trade differently depending on our P&L, but we know that it’s not the case. The most obvious example is if we suffer a large drawdown—we will trade scared. If it’s due to a dumb mistake that we made, then we will be even more likely to trade in a fearful state. It’s difficult to trade well when you’re scared of your shadow. Thus, we should make that we never put ourselves in a situation where we are facing a big drawdown and behind the 8-ball. One solution is to be extra vigilant about our risk limits and stops so that we get out of losers quickly. The second is to trade smaller position sizes, so that a string of losers won’t cripple us. The third is to change our strategy, so that we are employing a strategy that won’t put us at risk of a crippling drawdown. The definition of what constitutes a catastrophic drawdown will vary depending on the individual involved, but the bottom line is that you need to know your own limits.

 One other psychological interaction involving luck and skill is our ego. If we are taking too much risk and thereby exposing ourselves to too much luck in the short term, then we are setting ourselves up for a wild ride. If we make a ton of money quickly, then we will think that we are geniuses and gods. If we lose a lot of money, then we will most likely despair. Either state would be destructive to our trading, because it would mean that we are not on an even keel. Ideally, we would always be as calm and detached as possible, making the most objective decisions possible. Practically, that is not possible. But we can preserve our ego from such swings by reducing risk. We need to keep risk small and focus on the longer term game of trying to grind it out and make good decisions. This is one of the reasons why most people trading smaller position sizes than suggested by the Kelly Formula—because it will keep our ego in check.

The balance between luck and skill is a delicate one. Hopefully this piece has given you several ideas about how to manage risk properly as to minimize the role of luck on your financial and psychological capital.

No relevant positions

By Bruce Bower | E-mail: Bruce [at] howoftrading.com

Blog: www.howoftrading.com | Twitter: @HowOfTrading