Backtesting in Clojure with ta4j

I want some tools in clojure to allow for backtesting/analysis of trading strategies.

This is a good example of the analysis i'd like to do. https://marketlife.discoursehosting.net/t/rsi-exploration/2411

They use Excel there; Clojure has to be better right`

So much English language is required to explain the spreadsheets. There are no intermediary named variables, just columns, and you can see reading that thread how awkward it is to describe and understand descriptions of that type of analysis.

Ta4j is a java library for technical analysis and trading strategy backtesting, and it's pretty good. It's got a lot of indicators and they seem fairly well tested and reliably correct. It allows trading strategies to be played around with without worrying about implementing many indicators and trading rules. It presents a good set of primitives for this type of analysis.

A very rough cut of how we might start to play with ta4j in clojure:

First we have to depend on ta4j: for leiningen:

[org.ta4j/ta4j-core "0.12"]

Or deps.edn:

org.ta4j/ta4j-core {:mvn/version "0.12"}

...
(:import [org.ta4j.core BaseStrategy BaseTimeSeries$SeriesBuilder Order Order$OrderType TimeSeriesManager]
         [org.ta4j.core.analysis.criteria AverageProfitableTradesCriterion AverageProfitCriterion BuyAndHoldCriterion LinearTransactionCostCriterion MaximumDrawdownCriterion NumberOfBarsCriterion NumberOfTradesCriterion ProfitLossCriterion RewardRiskRatioCriterion TotalProfitCriterion]
         org.ta4j.core.indicators.helpers.ClosePriceIndicator
         org.ta4j.core.indicators.RSIIndicator
         [org.ta4j.core.trading.rules CrossedDownIndicatorRule WaitForRule])
...

(def bars [ ,,, ]) ;; Maps of bar end time(ZonedDateTime)/open/high/low/close/volume

(def series
  (let [s (.build (org.ta4j.core.BaseTimeSeries$SeriesBuilder.))]
    (doseq [{:keys [open high low close vol dt]} bars]
      (.addBar s dt open high low close vol))
    s))

(defn rsi-strat [series period oversold-thresh hold-periods]
  (let [rsi (RSIIndicator. (ClosePriceIndicator. series) period)
        entry (CrossedDownIndicatorRule. rsi oversold-thresh)
        exit (WaitForRule. Order$OrderType/BUY hold-periods)]
    (BaseStrategy. entry exit)))

(defn go [series strat]
  (let [mgr   (TimeSeriesManager. series)
        rec  (.run mgr strat)
        crits [(AverageProfitableTradesCriterion.)
               (AverageProfitCriterion.)
               (BuyAndHoldCriterion.)
               (LinearTransactionCostCriterion. 5000 0.005)
               (MaximumDrawdownCriterion.)
               (NumberOfBarsCriterion.)
               (NumberOfTradesCriterion.)
               (RewardRiskRatioCriterion.)
               (TotalProfitCriterion.)
               (ProfitLossCriterion.)]]
    (into {}
          (for [crit crits]
            [(-> crit .getClass .getSimpleName)
             (.doubleValue (.calculate crit series rec))]))))

(go series (rsi-strat series 14 30 x))

After after a bit of this I wanted to try to cut down on a bit of verbosity as quickly as possible, so I did this:

(ns mylib.ta4j
  (:import [org.ta4j.core BaseStrategy BaseTimeSeries$SeriesBuilder]))

(defn ind-values
  ([ind] (ind-values (-> ind .getTimeSeries .getBarCount) ind))
  ([n ind]
   (->> (map #(->> % (.getValue ind) .doubleValue)
             (range n)))))

(defn constructor [pre-str post-str]
  (fn [class-key args]
    (let [kns       (when-let [x (namespace class-key)] (str x "."))
          class-str (str pre-str kns (name class-key) post-str)]
      (clojure.lang.Reflector/invokeConstructor
       (resolve (symbol class-str))
       (to-array args)))))

(defn ind [class-key & args]
  (let [ctor (constructor "org.ta4j.core.indicators." "Indicator")]
    (ctor class-key args)))

(defn rule [class-key & args]
  (let [ctor (constructor "org.ta4j.core.trading.rules." "Rule")]
    (ctor class-key args)))

(defn crit [class-key & args]
  (let [ctor (constructor "org.ta4j.core.analysis.criteria." "Criterion")]
    (ctor class-key args)))

(defn ->series
  "Bars should be a sequnce of maps containing :end-zdt/:open/:high/:low/:close/:volume"
  [bars]
  (let [s (.build (org.ta4j.core.BaseTimeSeries$SeriesBuilder.))]
    (doseq [{:keys [end-zdt open high low close vol]} bars]
      (.addBar s end-zdt open high low close vol))
    s))

;;todo: other constructor signatures
(defn base-strategy [entry-rule exit-rule]
  (BaseStrategy. entry-rule exit-rule))

With reflection we can take advantage of the naming schemes in ta4j and cut down on some redundancy. We also now don't need to :import every ta4j class we use.

Now we can just go like this:

(let [series (->series data/spx-bars)
      rsi    (ind :RSI (ind :helpers/ClosePrice series) 14)
      strat  (base-strategy (rule :CrossedDownIndicator rsi 30) 
                            (rule :WaitFor Order$OrderType/BUY 5))
      crits  [(crit :AverageProfitableTrades)
              (crit :AverageProfit)
              (crit :BuyAndHold)
              (crit :LinearTransactionCost 5000 0.005)
              (crit :MaximumDrawdown)
              (crit :NumberOfBars)
              (crit :NumberOfTrades)
              (crit :RewardRiskRatio)
              (crit :TotalProfit)
              (crit :ProfitLoss)]
      mgr    (TimeSeriesManager. series)
      rec    (.run mgr strat)]
  (into {}
        (for [crit crits]
          [(-> crit .getClass .getSimpleName)
           (.doubleValue (.calculate crit series rec))])))

This may not be a perfect "library" but it feels nicer to me for now and can always be improved later.

The ta4j wiki has some more ideas of what you can do from here, and it may help or be necessary to delve into the source a bit.

Ta4j is a nice high level technical analysis library that we can quickly take advantage of with clojure interop. But I'm left feeling like something is missing. The data is too opaque.I don't want to keep looking up the object hierarchy and getters and setters. Clojure has spoiled me, I'm used to being able to manipulate this type of data using the core library functions without learning much of anything. I'd like to be able to drop the first hundred items of the time series and then get the closing price of every third bar that occurs on a Tuesday, sum the results and add 42 without having to refer to library specific documentation. I want a trade to be a map that prints out and tells me it contains an :entry and :exit order and know what those are, instead of having to look up what a trade is and what methods i can call on it.

The things we're doing here: taking time series data and applying indicators to them which are just functions, and then running a strategy, also just a function that accumulates some context such as trades, which themselves might have some local accumulating state. Ultimately we end up with a sequence of results of applying the strategy to our time series. My point is we don't really have to leave clojure.core to have a pretty decent backtesting framework in clojure!