How to Avoid Coding a Ghost Trader

In many cases past trading performance is an important indicator. If we are constantly trading a single instrument, this is easily derived by looking at the equity curve. However, this is no longer true, if we either

  • start and stop trading, e.g. after losing/ winning a number of trades in a row, or
  • trade multiple instruments simultaneously as a portfolio.

What we need in these cases is a ghost trader, providing us information how well our strategy would have worked - if we were actually that specific instrument enabled.

Programming a ghost trader is a lot of work. Ultimately, this amounts to re-implementing the backtesting engine in most of it’s complexity: we need to keep track of positions, price action, entries, exits, and safety stops. What’s even worse about it is, that all this code is highly dependent on the strategy. Most likely, we need to adjust it whenever the strategy changes and run a high risk of the ghost trader being a poor representatative of our strategy’s performance. This all amounts to a simple outcome: it’s best to avoid coding a ghost trader and rely on the backtesting engine instead. In the following I will outline how this can be done with MultiCharts in EasyLanguage.

With this proposed approach to trading algorithms, we can distinguish the following operating modes of operation:

  • 'good enough' single-cycle backtesting
  • automated trading
  • precise' backtesting

The single-cycle backtesting is important during strategy development and optimization. It allows us to tweak our strategy and see the results immediately. This is an absolutely must-have mode as our productivity relies on it. This mode is characterized as follows:

  • work with a very high amount of equity, e.g. $100,000,000
  • trade the full amount of shares for active instruments
  • trade just 1 share for inactive instruments

As we keep trading instruments we shouldn’t, there is a discrepancy between what we are simulating and what we wanted to simulate. This discrepancy is inverse propertional to the ratio between the equity we are simulating with and the price of 1 share of the inactive instruments. This is why we are backtesting with a huge amount of equity.

During automated trading, we don’t want to put in these additional trades. For once, because they most likely have a negative effect on our trading performance but also, because the total number of instruments we can trade is typically limited by the broker. The workaround is simple:

  • save per-instrument performance data to disk during “good enough” backtesting
  • load these data from disk and use them for trading decisions during automated trading

As EasyLanguage does not provide a way to save data to disk, we make use of the free ELCollections library.

The precise backtesting is essentially identical to the single-cycle backtesting. Unlike the single-cycle mode, we no longer need to apply huge amounts of equity and we won’t need to keep the instruments trading.

Now to the implementation details.

The money management signal is responsible for making portfolio decisions and enabling/ disabling the individual instruments. Instead of relying on pmms_strategy_allow_entries and pmms_strategy_deny_entries, I am using pmms_set_strategy_named_num to assign equity to the individual instruments. The relevant code looks like this:

for stockIdx = 1 to pmms_strategies_count begin
    if stockIdx <= stocksToTrade then begin
        // top stocks handled here
        pmms_set_strategy_named_num(stockIdx - 1, "MAX_EQUITY", Equity.PerStock);
    
end else begin
        // remaining stocks handled here
        if GetAppInfo(aiStrategyAuto) = 0 then begin
            // keep underperformers at 1 share to observe performance
            pmms_set_strategy_named_num(stockIdx - 1, "MAX_EQUITY", 0.01);
        
end else begin
            // exact: shut-off underperformers
            pmms_set_strategy_named_num(stockIdx - 1, "MAX_EQUITY", 0);
        
end;
    
end;
end;

In the trading signal, the value is received and used like this. We keep NumSharesToTrade at a minimum of 1, while we have been assigned any equity at all:

MAX_EQUITY = pmm_get_my_named_num("MAX_EQUITY);
NumSharesToTrade = MaxList(MAX_EQUITY / close, iff(MAX_EQUITY > 0, 1, 0))

Now the signal can trade as usual. We observe the trading activity with this code - which is completely independent from the actual trading strategy:

variables:
    intrabarpersist LastPositionsAgo (1),
    
thePos                           (0);
while(LastPositionsAgo < MaxPositionsAgo) begin
    thePos            = MaxPositionsAgo - LastPositionsAgo;
    
LastPositionsAgo += 1;
    
value2 = ExitPrice(thePos) - EntryPrice(thePos) - 2 * commission;
    
NetProfitObserver += NumSharesObserver * value2;
end;
if MarketPosition = 0 then begin
    NumSharesObserver = Round(100000 / close, 0);
end;
pmm_set_my_named_num(“NET_PROFIT”, NetProfitObserver);

It is important to notice that LastPositionsAgo must be of type intrabarpersist. NumSharesObserver holds the number of shares for our observer purposes: To create an indicator for trading performance, we pretend to continuously trade $100,000.00. The trading activity will then accumulate in NetProfitObserver. We provide this value back to the money management signal.

At the first bar of the day, the net profit is saved to a collection, using the previous day as the key. The collection is written to disk on the last bar, so that it can be reloaded later:

once begin
    // create directory to save CSV files
    if not(ELC.PathExists(dirName)) then ELC.DirectoryCreate(dirName);

    
// create map element for each strategy, read current data from disk
    for stockIdx = 1 to pmms_strategies_count begin
        collection.mapID   [stockIdx] = MapNN.New;
        
collection.filename[stockIdx] = dirname + "\"
                + pmms_strategy_symbol(stockIdx - 1) + ".csv;
        if ELC.PathExists(collection.filename[stockIdx]) then begin
            value99 = MapNN.ReadFile(collection.mapID[stockIdx],
                    collection.filename[stockIdx]);
        
end;
    
end;

end;

if date > date[1] then begin
    for
stockIdx = 1 to pmms_strategies_count begin
        if GetAppInfo(aiStrategyAuto) = 1 then begin
            // get net profit from collection
            value1 = MapNN.Get(collection.mapID[stockIdx], date[1]);
        end else begin
            // get net profit from strategy (and write to collection)
            value1 = pmms_get_strategy_named_num(stockIdx - 1 “NET_PROFIT”);
            value99 = MapNN.Put(collection.mapID[stockIdx], date[1], value1);
        
end;
    
end;
end;

if LastBarOnChart and GetAppInfo(aiStrategyAuto) = 0 then begin
    // collect last values and save collection to disk
    for stockIdx = 1 to pmms_strategies_count begin
        value1 = pmms_get_strategy_named_num(stockIdx - 1 “NET_PROFIT”);
        value99 = MapNN.Put(collection.mapID[stockIdx], date, value1);
        
value99 = MapNN.WriteFile(collection.mapID[stockIdx],
                collection.EBound.filename[stockIdx]);
    
end;
end;

As the code only stores performance data during single-cycle backtesting and on a daily basis, our automated trading routine will work as follows:

  • perform backtesting until end of day today and save performance data to disk
  • perform automated trading tomorrow, using the previously saved performance data
  • shutdown at the end of the trading session and repeat

I perform this daily routine with the help of an AutoIT script, because MultiChart’s recalculate command cannot be called from Portfolio Trader’s money management signals.

Now all of the above is only true in Synchronous Autotrading (SA) mode. In this mode, the market positions on the chart are forced to match the market positions at the broker. The history of positions will start at the point where we transition from the past (backtesting) to the present (autotrading). This is why our above observe code won’t work and why we had to save the data to disk.

 In Asynchronous Autotrading (AA) mode, the situation is different. The market position on the chart is now independent from the market position at the broker. At the time we transition from the past to the present, the market position can be forced to match though. Here is a screenshot of the strategy properties:

strategy properties

Now our observer code will see all the trades and continues to deliver performance data. As our equity is probably not infinitely high, we want to observe sending out the single-share orders. We can do that by making a slight change to our money-management code. This change will keep the observer running during backtesting, and turn it off for today:

for stockIdx = 1 to pmms_strategies_count begin
    if stockIdx<= stocksToTrade then begin
        // top stocks handled here
        pmms_set_strategy_named_num(stockIdx - 1, "MAX_EQUITY", Equity.PerStock);
    
end else begin
        // remaining stocks handled here
        if date <> CurrentDate or GetAppInfo(aiStrategyAuto) = then begin
            // keep underperformers at 1 share to observe performance
            pmms_set_strategy_named_num(stockIdx - 1, "MAX_EQUITY", 0.01);
        
end else begin
            // exact: shut-off underperformers
            pmms_set_strategy_named_num(stockIdx - 1, "MAX_EQUITY", 0);
        
end;
    
end;
end;

There is a catch though: with asynchronous autotrading the market positions on the chart and at the broker might start to deviate. We need to make sure we take proper action in case they do. Here is a possible solution:

if fancyCondition
    and MarketPosition > 0
then begin
    sell ("Signal LX") next bar at market;
end;

if fancyCondition
    and MarketPosition_at_Broker > 0  // we *have* a position...
    and MarketPosition = 0            // but our strategy think's we're flat
then begin
    sell ("Sync LX") next bar at market;
end;

Even though this works fairly well, you will probably want to keep the code for the ‘exact’ mode. I’ll leave it to you to figure out under which conditions the various little code snippets need to be enabled or disabled.

Happy coding!

© Felix Bertram 2002-2015. Last update September 2016.