There have been a few defining moments in my trading journey. One happened on a warm summer afternoon when I was really struggling with my trading and was close to giving up. I was talking with a friend of mine who was a former CBOT Bond trader turned broker and expressed frustration with my trading results. His response was simple. He said, “It’s not working for you because markets don’t move like that.” I asked the obvious, “Well, how do markets move?” He thought for a minute and said “I guess I’m not really sure I can answer that question, but not like that.” Very helpful, huh? I thought about that conversation constantly for the next few days and realized that I either had to answer that question or stop trading altogether. This launched a journey for me that consumed thousands of hours of my life, eventually sent me back to school to get an MBA and do a ton of PhD level coursework in math and quantitative techniques, and then to NYC where I now trade with these cool guys at SMB. All in all, it’s been a fantastic journey, but I’m sure Doug had no idea how much his one simple statement would change my life over the next decade.

One of the most significant “take aways” from my journey has been that you do not have to believe what people tell you about markets. The data is there, available (at least on a daily time frame) for free from Yahoo! Finance, and Excel is probably on 99% of the computers in existence. You can learn to think about trading problems in a scientific, mathematical mindset and answer many important trading questions for yourself.

A few important caveats are in order (and please, these are very important). First, you must have a good understanding of your tools. For instance, do you know the difference between a mean and a median and why one might be more appropriate in certain cases? Do you know when to use an arithmetic, harmonic or geometric mean? Why should nearly all trading ideas be thought of in terms of returns rather than absolute price changes? When might a percentage be misleading and you would be better off using a case study approach? These are basic issues in data analysis, and if you cannot answer each of those with complete confidence then you need to be careful. If you charge forward and do a lot of research without really understanding the techniques you are applying, you are apt to get misleading answers. (The good news is that this education is readily available and can be done mostly on your own if you apply yourself. Some schools (for example, MIT) offer many relevant classes online and for no charge.)

Secondly, framing the question correctly is at least as important as doing the analysis correctly. I believe (and I’m probably going to catch a lot of flak over putting this in writing) that one of the greatest dangers in modern finance is the army of engineers, physicists and other people with deep analytical backgrounds but no actual trading experience. Anyone with a good math background can get a CQF and get a job in the field, and, at least to me, this is a serious problem. I would much rather have a trader who has burned through a couple small E*Trade accounts trying to learn to trade than a guy with three advanced degrees who doesn’t understand why it’s a little difficult to apply his system to a $20 stock that does 10,000 shares/day and has a $0.40 spread. The disconnect between theory and reality is something that can’t be understood without actual trading experience. As Nassim Taleb says: “Read every piece of theoretical research you can-but stay a trader. An unguarded study of lower quantitative methods will rob you of your insight”, or “Never let a day go by without studying the changes in the prices of all available trading instruments. You will build an instinctive inference that is more powerful than conventional statistics.” You cannot ask the right question without some intuitive sense of how markets work, and you cannot get that sense without actually trading.

Lastly, remember that the answers you find are not absolute truth. Certainly, there are limits to what can be inferred from statistics, and even if you find something fairly concrete, it is only valid for the specific data set you were looking at.  Maybe the answer is different on different markets? It’s also possible (read: absolutely certain) that you will have made some mistakes in framing the question and have overlooked some obvious issues. This is why bouncing your ideas off of other smart traders can be a very, very good idea. Sometimes you will explain something to someone and they will tell you you made a stupid, obvious mistake. Collaboration is very important in this business. (As an aside, I’ve probably learned more from my mistakes (which are legion) than anything else over the years.)

I’ll wrap this post up here, but over the next couple of posts, which will hopefully happen this weekend, I will show you one way to look at the important question of what happens when a market gaps up or down on the open. Every trader should have a basic knowledge of market statistics, and this is a good place to start.