Reviewing the RegimeProcess
An evaluation of the project's development, testing, and literature
The first entry published here on zanit.me detailed the development of an experimental stochastic model for the CBOE Volatility Index ($VIX). While I hope that the substance of this experiment was of some interest, the time that I’ve spent investigating the project’s code since its release has revealed a complex reality. In this entry we’ll take a step back and get a wider look at the RegimeProcess. I’ll address the project's design decisions, the sources of its faltering rigor, and we’ll see what lessons we can draw for future endeavors.
The RegimeProcess' goals
Understandably the RegimeProcess wasn’t developed for me to sell financial derivatives on some market. Instead, I intended to:
- Develop a master project: Derivative pricing is a synthesis of multiple subjects that I had some familiarity with, including stochastic processes, partial differential equations, and mathematical modeling. I wanted to use the opportunity to flex some proficiency.
- Write usable software: An explicit intention of the RegimeProcess was to be built as a feature on top of the QuantLib C++ library. QuantLib is open source, and the project required only a basic amount of plumbing into its source code.
- Finish as quickly as possible: I wanted to complete a respectable version of the project and article as soon as possible. Additionally, I wanted to practice the trend of LLM-augmented development; the project wouldn't really have been viable without it.
- Document and publish the experience: In addition to the usual README.md file and public repository, I wanted to make equally available the rationale that directed the project and its design decisions.
What worked
The RegimeProcess had some marked successes to look back on. The synthesis of domains described above was, for me, ambitious and undeniable. In particular I look to:
- The execution: The project consisted of the core RegimeProcess class, two Monte Carlo engines, Python bindings, the particle filter, an expectation-maximization (EM) algorithm, the pricing notebooks, and multiple levels of abstraction. These moving parts did come together to execute “free” of runtime errors.
- The timeframe: If I remember correctly, I believe this project was completed in around three days. It impressed me how much was done considering the project's waves of engineering challenges.
- The experience: The project required the consideration of several famous stochastic processes such as Ornstein-Uhlenbeck, Cox-Ingersoll-Ross, Heston, and Bates. I was thrown headfirst into the worlds of Monte Carlo simulation, expectation-maximization, and quite urgently, \(P\) and \(Q\)-measure pricing.
What didn't work
The reality of the RegimeProcess is that the outcomes described above smoothed over numerous logical issues. In addition, the developmental constraints I adhered to were able to produce a product, but had several flaws. This includes:
- Market data calibration and pricing: The uncompromising reality is that a process dependent on a stateful and agnostic set of subprocesses, each with their own evolution mechanisms, is bound to have significant complications—in particular:
- Object interfaces that were naturally mismatched with QuantLib’s tool suite required the hardcoding of several algorithms, and introduced significant overhead when defining parameters.
- Inferring regimes at each step of historical data is a task that can square the amount of computational workload.
- LLM-augmented development: One can memorize, but not truly internalize, the fact that LLMs cannot be relied upon to cover up missing expertise. Heavy LLM usage allowed this project to be synthesized at speed, but engineering violations such as those above went unnoticed without explicit errors or human foresight.
- The article: I was excited at the prospect of packaging these concepts into a neat, attractive article. In reality, such writing takes acquired skill and a true command of domain knowledge. I fear that what I produced was intimidating to some, and inaccurate to others. It made me reflect on the ethical question of this journal’s true intentions; I try to express the answer to that on the journal’s homepage.
Conclusions
The 90-90 rule comes to mind:
“The first 90 percent of the code accounts for the first 90 percent of the development time. The remaining 10 percent of the code accounts for the other 90 percent of the development time.”
- Tom Cargill, Bell Labs
Even as a valuable learning experience, the RegimeProcess became a cautionary tale. I believed that I had fully accounted for issues such as runtime errors, numerical instability, and conceptual obscurity. Because of this, the project came to life in a respectable timeframe, but when the scope of its engineering challenges escaped my grasp, I glossed over them to adhere to schedule.
I told myself at the time that I was realistically prepared to take on this challenge. Indeed in a low-stakes situation like that one, a common rule of thumb is to simply go ahead and try. In doing so, then explaining the project, and then finally reviewing it, I learned some valuable lessons about derivative pricing, software development, and myself.
The RegimeProcess is not dead.
Comments
Post a Comment