How to Beat the Stock Market: An Extensive Guide
If you want to learn how to beat the stock market, you simply need a plan, and today I’m going to give you one. First off, there are traders, people active in the markets. Then there are investors who know they can outperform mutual funds (I seriously think a monkey could outperform most mutual funds by randomly picking stocks.) This guide is for traders and investors alike, read on to the end and you’ll see why.
WARNING: Around the globe, people are diving into the stock market because they think it's an "easy" way to make a lot of money fast. This business is definitely NOT a get rich quick scheme. It takes YEARS of hard work and dedication to get it right. If you think you can spend a couple hours reading a book on trading then become a millionaire, you're dead wrong and you will lose all your money. What I'm about to share is a fairly extensive guide that took me 10 years to develop, but that's the amazing part. I can distill my 10 years of experience into a guide that will rapidly increase your learning curve. Even with your accelerated learning, this is still going to take work, it's not a hobby, it's a profession.
Now for some good news: YOU CAN DO THIS!!! As you read through this don’t get discouraged and say, “This is way too hard.” I have some great news at the end that will have you ecstatic how you approach the markets.
The Stock Market: Where Logic and Reason Thrive…haha yea right
Let me tell you a little secret: the stock market is full of horrible investors. Most people have little to no plan when it comes to their trading and investing. They follow advice from talking heads on CNBC such as Cramer or Fast Money (Google the performance of Cramer and Fast Money), or other pseudo-sciences such as Elliot wave theory or technical analysis. There are even the cycle-theorist that claim the moon or presidential cycles can make them rich.
When it comes to money, people get emotional. When people get emotional it’s the exact time they shouldn’t be making financial decisions, but that’s exactly what happens in the stock market.
I’ll get into all the reasons why these are incredibly dangerous ideas for your portfolio later. But I must come clean first; I was once one of those terrible investors myself.
Early in my trading and investing career I could be found on all the forums, the “guru” websites, reading decade old books on technical analysis and drawing trend lines all over my charts.
How many winning trades did you have last year?
Now, I did make money…but about less than half the time. I remember watching my trades go from good to bad. Or from good to great and then I didn’t want to sell and when I did the market keep going, and I would chase after it. I don’t even want to think of all the money I burned on commissions themselves. I remember saving up for months to deposit more and more money into my brokerage account just to watch it evaporate.
If you’re a trader, I’d like you to be honest and tell me how many winning trades you had last year. Do you have a spreadsheet, are you keeping track? The point is how do you know how well you’re doing? I finally started to log my trades and was stunned to see I was right less the 50% of the time… and that’s when it hit me… I had NO idea what I was doing. Talk about a hard thing to internalize. Thousands and thousands of dollars lost and years of keeping notes, watching TV and researching fundamentals, all for less than nothing.
I’ve always been a skeptical and analytical person, so I finally started to look around for something that could be proven to work. As an electrical engineer, I was always taught to break a complex system down to a more simplistic set of models.
Let’s create a system to beat the market
And that’s precisely what I set out to do, create a model of the stock market and use it to tell me when to buy and sell. Here is what I developed in 2012 after years of work:
Now that’s how to make money in the stock market! I encourage you to read on and see how I developed this system and how you can too. Your portfolio will thank you.
Part 1: Build a set of rules to beat the stock market
At its basic level, a trading model is a set of defined rules that give you definite buy and sell signals for the stock market. Technically you could do this by hand if your rules were of the utmost simplicity, such as “buy a ten-day high” or “sell a ten-day low.”
You could pull that off OK. But with today’s computing power and an incredible amount of “Big Data,” it’s a lot easier to just use a computer. With today’s widespread availability of computers, anyone that does not take the time to test their rules of the market is just being plain lazy.
The Hardest Part of Beating the Stock Market
Coming up with your rules is really hard at first. I can’t really explain how it happens but when you look at this stuff all the time you start to ask yourself questions like following:
- How many days is the stock market above the 5-day moving average and how many below?
- What percentage does it move away from that average to the upside, what about the down?
- How can I know when a pullback is over?
- How can I determine if the momentum will keep pushing us higher?
- What if I buy at 50% above yesterdays high vs 49%? What’s the result?
- What happens when I buy if the RSI is below 30?
- What happens if I sell if the RSI is below 30?
These are the types of questions you can turn into defined rules and then you can test them…more about testing later.
Try asking yourself or your stock picking guru exactly how they got their ideas, and we’re talking EXACTLY how. Then ask them to show their proven track record.
I’m guessing not many will be able to do so. Again, the basic premise of an investing system is to use past data to determine the likelihood of something happening in the future. Here is an example of the system I use to trade the stock market showing the exact buy and sell points:
Eliminate your emotions if you want to beat the market
You will notice that on the far right is the “Brexit” crash of 2016. Everyone was throwing the baby out with the bath water. But a mathematical, statistically tested system does not have emotions, it only has cold hard math behind it (cold hard math gives me a warm fuzzy). It bought right as everyone was selling. Remember the horrible and emotional traders mentioned previously? It took their money and ran all the way to the bank…and so did I, I might add.
Part 2: Choose a stock market theory to beat the market
“The past does not repeat itself, but it does rhyme.”
– Mark Twain
Past data is used to produce a working model of what a system will likely do in the future. Galileo, Newton, and Kepler used observations of the planets to produce models of how they moved in orbits and then confirmed their models with real-time observations.
That’s precisely what we do when building a trading system. We look at past market data and create stock market theories based on mathematical principals. We then use those theories to make a system to buy and sell in real time.
Stock Market Theories
We built a set of rules in part 1. Now it’s time to test those rules against a theory. Choose a theory for how to trade the markets and stick to it. I use several different theories from mean reversion, trend following and also momentum. This alone will save you thousands in losses by understanding which theory (currently…because they change) drives the security you’re looking at.
For example, many traders like to look for the “reversal.” A security has tanked and momentum is to the downside…then we see several days of consolidation. That trader may have a set of rules in place for when to buy: he places his order 1 tick above the highest high of the consolidation period because that’s his rule. A big move happens! The market takes off to the upside, only to come crashing down and then continues to move south. Did the trader set a stop loss rule? Are they now working on hope as a strategy?
This is what’s known as a “false break” and it happens all the time. What did the trader do wrong? Well, lots, but the simple answer is he was guessing that mean reversion was going to take place when momentum was clearly in control. Confusing your theories will lose you money.
A Sound Theory for a Sound Market: Mean Reversion
You may have noticed on the image above I titled it, “The power of mean reversion.”
The US Stock Market is a Complex Mean Reverting and Trend Following Data Series
Have you noticed how the S&P 500 loves to chop around? Up one day, down the next, like a seesaw. This is called “mean reversion” which is a complicated way of saying the market moves above and below its mean (or average price).
The S&P 500 is a mean reverting market
The S&P 500 is the tail that wags the dog. It’s the market that controls the world. Now with over $140 Billion traded daily. Compare that to the New York Stock Exchange at $4.5 Billion. The point is that there are millions of people around the world each playing their own game in the S&P 500, the result is that there is a ton of mean reversion going on.
Now, you may have noticed we were on a bit of a tear from late 2016 through 2017 and it seems like we just went up up up. Well, that was an anomaly of the S&P that does happen from time to time where there is very little mean reversion, instead, the market follows a trend, higher and higher.
Trend Following: another sound theory
Have you noticed that the stock market also loves to move in one long direction for an extended period? This is called “trend following” which is a complicated way of saying a bull or bear market. Here’s an example of our Gold Futures Trend Following System when gold went on a tear with over $10k in profits on this trade.
There are no easy answers on how to beat the stock market
When you put these two market observations together and characterize them in mathematical terms, you have a working model for how the S&P 500 works.
Sometimes the S&P is very choppy and mean reverting. Other times it goes in long streaks, trending higher. Usually, it’s a mixture of both. There is no right answer here. If everyone knew what the market was going to do, we’d all be millionaires, right? The point is that there are no easy answers in this business. Sorry get-rich-quickers…not a chance.
There are other markets that are more trendy, like the Russel 2000, or most currency markets. You simply have to do your homework on what market you’re trading. Most stocks are trend-based first with mean reversion mixed in. The S&P 500 is mean reversion-based with some trending mixed in.
Part 3: Chicken/egg your rules and theories to create a system that beats the stock market
Again, no easy answers. You’re going to have to bounce back and forth between your rules and your theories and markets to figure out what works best for you. Some people love trend following because you swing for the fences with HUGE wins but you lose 75% of your trades. Other’s like the base hit wins of mean reversion where you’re winning 80% of your trades but for small amounts on each win.
But enough about that now I’m going to deliver what has worked for me. Remember that P/L chart showing the $1M in profits from the beginning it’s all based upon this:
Buy Pullbacks in Bull Markets and Sell Run-Ups in Bear Markets
I trade the S&P 500. The S&P 500 is a mean reverting market first and trending second. I built a set of rules based on mean reversion and trending, then I applied those rules to the S&P 500
If the S&P 500 is in a bull market (the trend) and the market has pulled back below its average (mean reversion): BUY.
Once the market has moved back to its average: SELL. If done correctly this can be done with 80% accuracy.
I’d like to point something out, you will rarely buy right at the bottom, nor will you sell at the top. This alone has lost traders millions of dollars trying to “time” their trades perfectly. Give it up folks, you’re chasing clouds.
After years of work, this system does a pretty good job of getting near the bottom, but notice it never holds to the top. I’ll reiterate what I said above the image: Once the market has moved back to its average: SELL. We don’t look for it to go beyond its average, just back to it.
What about dem Bear Markets?
If the stock market is in a bear market and the market has risen above its average: SHORT. Once the market has moved back to its average: COVER. If done correctly this can be done with 72% accuracy. Here’s what the system looked like back in 2008…a time we all remember so fondly. I say fondly because this system pulled in over $30k in profit in 2008…how did your Certified Financial Planner do?
Bear Markets are like your crazy uncle: they move around a lot but generally, they are going downhill
A couple of notes: wow, that’s a lot of trades. Bear markets bring volatility. Increased volatility means more mean reversion. The more volatile the market, the more chances for trades. Don’t be scared of bear markets…treat them as what they are, an opportunity to trade more and make more!
Now, why the two different percentages? Why 80% on the upside and only 72% on the downside?
We all know that since the dawn of the stock market (with a few exceptions: the 1929 crash, the great depression, the 1987 crash, the tech-bubble and of course the great recession) the market has gone higher and higher.
As such, there is much more time in bull markets (more data) than in bear markets.
When you have less data, it’s harder to build an accurate model and thus a highly precise system.
Part 4: The devil is in the details
“Details matter, it’s worth waiting to get it right.”
– Steve Jobs
Adding to the challenge of developing any investing system are three major details that, if overlooked, will doom your portfolio.
The amount you pay your broker to execute a trade. With modern computerized order routing and the elimination of open out-cry of exchanges from yesteryear, you should be paying about $0.01 a share or around $2.50 for a futures contract. This seems very low, and it is, but over time these commissions add up to a lot of money. You must always account for commissions when developing a system.
The little-known fact that when you place a market order, you can’t buy precisely at the “bid” (the price someone is offering to buy at) or sell exactly at the “ask” (the price someone is offering to sell). There is always a slight difference in your fill price that also, over time, adds up to a lot of money.
That’s why we only focus on the deepest, most liquid markets such as the SP500 where the bid and ask are as close as mathematically possible.
FOREX and options will destroy your account just due to slippage alone!
This is one of the biggest oversights in trading and investing and also in almost everyone’s daily life. As humans with fantastic pattern-recognizing brains, we are always being fooled by randomness. Proving statistical significance ensures that something we observe is not random and can be used as real information.
As an example, say that you flip a coin five times and all five times it comes up heads. You might say to yourself “wow this coin is a heads only coin!” or “every time I flip, it just comes up heads!”
You might even think it’s weighted or “rigged.” However, if you were to keep flipping it, say 1000 times, you would see that you would have an almost exactly even number of heads and tails.
This Sub Title is Totally Random
Such it is with trading and investing. If you only have a handful of “oh look, when this happens, then that happens” randomness is probably fooling you. However, if you have hundreds of examples of “see when I do this, this happens” you are probably on to something.
People are fooled by randomness every day. From football games and a person’s “lucky” shirt to predicting election results base on only one or two previous elections. Again, when it comes to putting a dime at risk, you want hundreds and hundreds of examples of your theory.
The system I use has nearly 300 examples, which is highly statistically significant.
Further reading on the subject:
Harvard Business Review: https://hbr.org/2016/02/a-refresher-on-statistical-significance
Part 5: Testing, Testing, Testing: the only way you will KNOW you can beat the stock market
“It’s not that I’m so smart, it’s that I stay with the problem longer.”
– Albert Einstein
We have our rules, our theory, our market in place and we are accounting for the commissions, slippage and statistical significance. How do we know the system will work in real time with real money?
Testing, testing, and more testing.
This is about to get a bit technical, but I am a perfectionist, and you should be too when it comes to putting a single dime of your money at risk with any trade or investment.
Testing Method 1: Back Testing
With the trading system defended it’s time to run it on ALL the data you can find, however, there is a caveat. You want to reserve some of that data for later, say 20% of it. The dataset that the system is run on is called “in sample” data, the 20% data saved for later that the system has not seen before is called “out of sample” data.
When you are developing your system, you initially test it on the in-sample data to get a feel for whether it works. Here is what the system looks like on data “in sample” from 1997 to 2007:
Once you are satisfied with your system on in sample data, you need to test it on that 20% of data that it has never “seen” before. This is data that it has never been tested with before. This is called “out of sample data” testing.
Here is what you want to see when you test your system on the data that is out of sample:
Looks like the in-sample data, correct? This gives confidence that the system has not been over curve fit; once it goes live in real time with real money, it will continue to work.
And since you are probably wondering, here is the REAL-TIME performance of the system since I started using it in 2012:
Precisely what we want to see. It keeps cranking out profits in real time.
Testing Method 2: Walk Forward Optimization is a critical component to keep beating the market
Walk forward optimization (WFO) testing is the subject of many books and papers in academia; you can learn as much or as little as you want about the subject. It is a massive field of predictive data analysis.
The basic premises is to take small “slices” of the data and run your system over it, log key statistics, then move forward and run the system on the next “slice” of data that overlaps a bit with the older slice that was just tested.
Like before, with the “in sample” and “out of sample” testing, each slice of data has its own smaller set of “in sample” and “out of sample” data. This is one of the most rigorous methods for testing systems that use predictive statistics on entities as dynamic as the stock market. WFO is incredibly complicated, and I encourage you to use Wikipedia or Google to learn more.
Here is TradeStation’s definition:
“Many trading systems fail in the real world because they are built and tested on the same set of historical data. TradeStation’s Walk Forward Optimizer aids in the mitigation of this problem by performing a set of “walk-forward” performance tests against as-yet-unseen market data, thereby simulating the unpredictability of trading a strategy under real market conditions.”
Luckily for you and me, TradeStation has a user-friendly WFO function already built in.
After setting the number of data slices and the percentage of out of sample data, the WFO is let loose on the system and gives pass/fail ratings for each permutation of data slice and “out of sample” data set.
Here is what the WFO report run over my SP500 system looks like (please zoom in if needed):
This gives an extreme level of confidence that the system will continue to work in the future.
Here is the finished product after testing. I’ve marked off where I used in sample data, then out of sample, then finally you see the real time results. This tells me unequivocally that this is a robust trading system that stands the test of time. Now I can be absolutely certain I can use this trading model.
Part 6: Risk Management is the key on how to beat the stock market
“One cannot take risk management too seriously.”
– H. Felix Kloman
Most amateur traders only think about how much money they could make, not how much they could lose. Professional traders flip that on its head. They are obsessed with risk to their portfolio. Risk too much on a bad trade or investment, and they are out of the game. If you employ proper risk management, you can make money while keeping the risk to your portfolio low. For every trade/investment made you should know precisely how much money is at risk and exactly how many shares or contracts to buy.
Use this formula every time you make a trade or investment:
Number of Shares = ( % Risk ) * ( Portfolio Size ) / ( Current Volatility )
Example: You have a portfolio size of $20,000.00, and you only want to risk 5% of it with anyone trade/investment.
My system runs on the SP500, which has the ETF (exchange-traded fund) ticker “SPY”. Current volatility is how much the market is chopping around. I’ve found that the average range (the close minus the open) of the last 40 days is a good measure of volatility (you’ll need a calculator for this part). Say the 40-day average range is 5 points.
( 5% ) * ( 20,000.00 ) / ( 5 ) = 200 shares
Increased volatility means you should risk less
Notice that with volatility in the denominator, you will be buying more shares when the market is calm and less when the market is insane. This will keep you safe from overexposing your portfolio to unnecessary risk.
Also, notice that with your portfolio size in the numerator, you will be buying more shares as your account grows which will compound your account rapidly. However, if you take a few losses, you will be buying fewer shares which help to reduce portfolio drawdown even more.
The power of the above equation cannot be understated. If you take anything away from this report, please memorize this risk management formula. The system I use has the above logic hard-coded into every trade it makes:
For fun, I ran the trading model with and without the risk management formula. Below are the results:
Now here is the same image from the start of this article using the money management formula:
From $200K to $1MM…that’s that power of putting data behind your investments!
Summary: How to beat the market
“Let’s get down to the brass tacks.”
How do you wrap up all this information, put a nice bow on it, and beat the market? By coding it into a single program that runs every day. This is the only way that I know of unless you love crunching numbers on reams of paper or spreadsheets. The code keeps the math IN and the emotions OUT. Coding our ideas into computers tell us if we would have made or lost money.
To beat the market, we need to combine:
● A specific set of rules for when to buy and sell
● A stock market theory or blend of several that we apply the above rules to
● Risk and money management
● Deducted commissions
● Deducted slippage
● Statistically significant sample size
● Out of sample testing
● Walk forward testing
That is a lot of ingredients and coding them all together is the only way to do it. I spend years learning different computer languages as an electrical engineer, so it didn’t take me a lifetime to code it all up. It just took what felt like a lifetime to learn about the above information after years of losing money with my trading and investing. Years of failure turned into years of research to now, years of profits. Here are the cold hard statistics of my S&P 500 investing system:
The green arrows are what I think are the most important stats when it comes to beating the market.
A coin that lands on heads 82% of the time
Over 82% of the time, this system is right. That’s what I mean when I say, “It’s not about the possibility of making money, it’s about the probability of success.” Every single trade I place, I know I have an 82% chance of getting right. Can the “gurus” say the same?
I get excited when I lose a trade
Another big factor discussed is risk management. This system has only lost 2 trades back to back….EVER! In 20 years. It gives me a warm fuzzy every time I lose a trade…why? Because I know that means the next one is going to be a winner!
A crash-proof system
Finally, and this is a big one, my cash is only in the market less than 22% of the time. That means I have a 78% chance of missing a crash. Buy and hold will MURDER your account. I can’t tell you how many times I’ve read or heard about needing to be on the lookout for the next big crash? Guess what, I don’t care, because my money is most likely sitting in cash, and I can sleep at night.
Join me and we can beat the markets together
Obviously, this is a lot of work and you may have thrown your hands in the air long ago saying I can’t do this? But here’s the thing. You don’t have to reinvent the wheel. I just showed you that I’ve done it all…why not just use the stuff I’ve already built?
I’ve been doing this for so long that I not only have my mean reversion system, but I’ve built trend following systems and momentum based systems as well. And ALL of them beat the stock market.
The systems I’ve built are for traders and investors alike. The S&P System you see here, it only trades about 15 times per year but pulls in over 22%. Another system I have is based on momentum and it trades just 4 ETFs each month…it also does over 20%. These 2 systems alone will take you less than 2 hours per year to use. These are great systems for investors, but I really get excited about my new stock picking system. Trading 4-15 stocks per month, I can pull in over 45% PER YEAR!