Han Dong @headhaancho and Prerak Sanghvi @PrerakSanghvi from Proof Trading were interviewed in episode 149 of the WebRush podcast. The podcast is a good insight into the decisions and experimentations that are required to build a high performance application. Fortunately, one of the decisions from Proof Trading was to pick AG Grid as their front end data grid.
Prerak is CTO of Proof Trading, and he has 15 years of experience building high performance trading systems and real time user interfaces.
Han is the lead designer and Web developer at Proof Trading.
Proof Trading was created by the founders and early developers of IEX. IEX is a trading exchange for individual traders. Proof Trading is an algorithmic trading platform for institutional investors.
Prerak explains that these systems "are built for high information density. .. And of course, we need the interface to be extremely responsive, because when the system is trading hundreds of millions of dollars worth of stock at any given moment, you can't really be behind 5 seconds or 10 seconds either, because you don't know where you stand and that could be a significant amount of exposure."
In addition to being responsive, the user interface needs be well designed to "get as much information as possible on the screen without wasting a Pixel, but also without losing the usability aspects, you don't want to absolutely get lost."
Proof Trading uses a Server Sent Events architecture with Redis as a central cache, a node.js server subscribing to Redis updates which streams the updates to the browser client.
Because of the algorithmic nature of the trading and the high performance processing required much of the processing is performed server side where the processing can be scaled easily and then pass complete records of data to the browser so that the browser makes the rendering decisions.
Han describes the data grid requirements that led to the selection of AG Grid. "you can get data up to the size of a million rows... These are 50 columns fields. These are very large kind of data sets. It's a huge amount of data that we're kind of dealing with when it comes to handling that amount of data, especially with additional layers of data manipulation on top. Things like filtering and sorting is very heavy, compute wise. And so a traditional grid just isn't able to handle that kind of load.
Just right off the bat, you are looking at grids that are implementing virtualization both in row and column direction. That's a must for sure. If you're actually rendering every single row, that's not even going to happen."
AG Grid was selected from a number of competing products, including an in-house grid which was initially created to keep the technological options open.
Han describes customisation, flexibility and speed as the early evaluation criteria. "We want to have maximum customization and maximum flexibility in terms of what we want to do." The early comparisons of data grids showed that AG Grid "was as sort of fast as you can get with the kind of implementation that they're doing under the hood." Making it "a very easy choice from early stage."
AG Grid supports easy customisation, Han describes it as "an option to turn on and off. And that goes along with a lot of AG grid features. These are just essentially an option that you toggle to have the thing that you want."
The initial prototyping with AG Grid helped support system architecture decisions. Han describes the process as "in the beginning, as you're doing this, testing the small amount of data, everything works on the client side, you don't have to do anything fancy to get a very smooth working grid."
Then as the data load increases architecture decisions have to be made to create a highly performant system. Han describes it as "being a question of where to put the data modelling. Is it in the client side? Entirely on the back end? And there's actually a middle ground somewhere between. Where you can put things in the web worker, in the worker thread on the client side. We kind of think of it as sort of three different tiers of option that we can fall back on each having taken more compute load away from the client side. We're currently actually in the middle ground right now. We are having a worker data model that does all the data calculations in the worker thread and then pumping the snapshots back into the client browser."
Proof Trading is a React based system, but even with React, the Proof Trading team still write some components in plain JavaScript to be as fast as possible. AG Grid supports this by allowing customisation with React, or Angular or plain JavaScript components.
Han describes the customisation of AG Grid as a high point of its implementation.
"I think like, what's so kind of great about AG Grid is the fact that it's so customizable. So many of its component parts are up for. You can plug in with your own implementation. Things like custom tool panel, context menu, everything that they do, they sort of open a side door and ask if you want to come along and they do that very nicely by providing a nice set of APIs for you to kind of just have a way to access the kind of tool sets that they are using internally. We find that even though we have built a lot of customization on top of AG Grid, doing it was very easy because everything we need to do that is provided."
The podcast goes into many more details around the creation of High Performance Fin Tech systems and is worth listening to. You can find it on the WebRush website. In addition Prerak wrote an extensive blog post describing the architecture and decisions.