Data Intellect
What is Backtesting?
When investors want to define or use a new trading strategy, they need a method to verify its expected performance. One method for achieving this is running the strategy against historical data and assessing how it would have performed; this is called a backtest.
In our experience the term backtest is used to mean different things depending on the context. This blog discusses testing of a trading strategy to determine its capability for profit making and associated risk profile, centered around running a series of queries against a dataset. It also discusses considerations for a technical backtest involving historic market data to replay a stream of data through a simulated trading infrastructure, which can be used to test both strategy profitability as well as system performance and capacity.
Trading Strategy Backtester
Recently, we have developed a trading strategy application that enables users to efficiently test ‘portfolios’ of instruments for profitability. In this context, portfolios are user defined tables and contain all of the necessary columns and parameters needed to execute the backtest (e.g., different sizes, different entry and exit times, different holdings, different strategies). With this in mind, we are making market assumptions such as our participation times will always occur without fail and that any positions bought or sold will have minimal or known market impact as this is all queried against historical data.
Schematic diagram showing the infrastructure of a trading strategy backtester
Input and Backtest
The workflow of this trading strategy backtester is illustrated in the schematic above. Input parameters are stored within portfolios for each instrument which are then run efficiently against user defined strategies to calculate a trade-in price; this is how an instrument is bought. For example, we could have decided to trade into a position using a VWAP (volume-weighted average price) based algorithm over a specific time period. This will comprise an examination of trade market activity over a specific time period to determine which trades are buys and which are sells by matching on associated quotes.
While a portfolio’s instruments are held, any holding conditions will change the position based on market events or activity. For example, an instrument’s price may fluctuate above or below user specified percent boundaries such that a sell will occur, realising any profit or loss that results from this.
Lastly, during the trade out phase, the portfolio’s instruments are sold utilising the same procedure as the trade-in calculation. This enables users to extend and combine strategies across the trade-in and trade-out windows.
Results and Analysis
Those portfolios that are successful can warrant more detail on their profit or loss-making. The results and analysis piece of the backtester can be used to expand the profit and loss (PnL) from a backtest through the required time period. This can be further used to evaluate how well each instrument performs in relation to the potential profit available in the market (unrealised PnL), and the profit that has already been gained (realised PnL).
The mechanism of this expansion will take an existing trade-in price for each instrument from a backtest result portfolio and match a trade-out price based on the bid or ask prices from the market data quotes. To create a timeseries visualisation, we do this for each time bucket over the whole backtesting window.
The output of the timeseries can take different aggregations. The first plot shows an expansion of a portfolio (that contains two instruments), where the total PnL and realised PnL have been added up across all instruments. We can also aggregate the output table to display the PnL paths for each constituent as shown in the second plot.
In both plots, intermediate trades were carried out to realise some profit during the holding window. This is illustrated by realised PnL, which closes out the multiday period by liquidating every position across both instruments, realising the entire potential profit.
To conclude the trading backtester, it was built with two goals in mind; to analyse trading strategies and also to provide a framework for backtesting from which future users can expand. Therefore, this application needs to be extensible to allow users to run different parameterised versions of portfolios efficiently and anticipate any errors through validation to ensure garbage isn’t created, or that misconfigured portfolios are exited earlier with minimal work done. This sort of backtester should also be fast and parallelisable, so we don’t have to wait long for results.
Technical Backtester
With a technical backtester, replaying historic data is usually the first step towards allowing you to backtest. This replay can be performed in a stream of data through a simulated trading infrastructure, e.g., historical data is presented to the process as if it was real-time data. This is well illustrated in the diagram below where the subject of the test is not the profitability of the trading strategy, but rather that the strategy performs as it should do within a live trading system.
Using this type of infrastructure allows for a more comprehensive backtest as well as a way for verifying the correctness, performance, and capacity of trading strategy use cases.
This provides the ability to play data in real time or at an accelerated rate. It’s crucial to be able to give simulated time as well. For instance, if the process implementing the strategy has a timer that fires every five minutes, it must fire every five minutes of simulated time rather than actual clock time (as if we are running faster than real time, which is usually the desire, then it will not be firing frequently enough). This can be implemented within data capture systems as such TorQ, where processes make use of .proc.cp[] which is set to system time by default but can be changed to deal with simulated time more easily.
The information at the bottom of the schematic is important because it illustrates how market orders and signals are pushed into a database so that end-users may analyse their correctness. Market orders will be analysed in real time and may trigger a trading signal either to buy or sell an instrument’s asset. This workflow allows market orders to keep within the available volume of an instrument that may be bought or sold before being sent into the simulated market. This is a mechanism used to send market orders back to the strategy engine in order to make any adjustments, such as changing the position, which may modify the strategy’s behaviour. With this type of replay infrastructure, we are making the assumption that our execution of trades is not affecting the market itself and therefore not affecting prices of subsequent trades.
This type of backtesting can be performed using TorQ, whereby the datareplay.q utility provides functionality for generating tickerplant function calls from historical data. We previously published a blog that included an example of this, in which we were backtesting a VWAP (volume-weighted average price) computation by retrieving historical trade table data over a time period.
Conclusion
A trading strategy is a well-defined plan that outlines the manner in which financial instruments are bought or sold as market conditions fluctuate. Backtesting is a method employed to evaluate the theoretical performance of a trading strategy using historical market data.
Backtesting a strategy may contain a number of related steps. It is usually a good idea to take a high level approach initially, to test the theory at a macro level, and to allow you to proceed with or discount various approaches. For initially successful approaches, a more fine graned approach is required, potentially all the way down to a technical backtest to validate the correct execution of the stratgey.
If you found this article useful and informative, or additionally if you have further questions feel free to reach out to the AquaQ team at: info@aquaq.co.uk
Share this:
