Some Thoughts on Processing Power

Developing trading strategies, especially when developing strategies for portfolios, is a constant battle: you will quickly reach a point where your productivity and your imagination is limited by the processing power you have at hand.

Here are some parameters of a strategy for reference:

  • number of instruments: 200
  • bar resolution: 5 minutes
  • session length: standard US stock session
  • years simulated: 5.5 (1,400 trading days)

Simulating this strategy leads to the following results:

  • memory use: about 4GB (which means youll need at leat 6GB to work)
  • simulation time: about 20 minutes on a dual core i5 @ 2.6 GHz

If you feel this is not too bad, then unfortunately there is more. We can distinguish the following cases for simulation:

  • backtesting: MultiCharts will mostly run this in a single thread. Adding more cores won’t speed things up.
  • optimization: MultiCharts will run one set of parameters per core. While that won’t get faster by adding more cores, multiple parameter sets will be run in parallel, effectively cutting down total time.

Looking at PCs I found that the speed of a single core is pretty much the same, regardless of the core being an i5, i7 or Xeon (and at comparable clock speeds). Or in other words: you can’t just go and buy a faster processor to speed up your backtesting. What you can buy is a machine that has more cores. This will speed up your optimization but won’t help with backtesting.

So, how many cores can you buy?

  • an i7 in a standard desktop comes with 4 cores
  • you can find some dual Xeon workstations (e.g. at Titan Computers):
    • a dual Xeon, Ivy Bridge, with 8 cores starts at about $2,000
    • a dual Xeon, Ivy Bridge, with 16  starts at about $6,000
    • a dual Xeon, Haswell, with 16 cores starts at about $10,000.
  • you can move your processing to the cloud. A 16-core machine (Ivy Bridge) runs for about $1.45/hour with Googles Cloud Compute Engine (including Windows licensing fee)

At least for me, it seems that running my optimizations in the cloud is the most cost-effective solution to get access to a machine with enough power for what I need to do. 

Now the work is split across multiple machines: one to develop at home, one to optimize in the cloud and one for live trading. This calls for a proper revision control system. I’ll talk about that in a later post.

© Felix Bertram 2002-2015. Last update September 2016.