Intro
Although this sounds not that intuitive, the performance of most modern hardwares are not being capped at clock speed, instead, it is the memory access speed.
And, just wanna let you know this passage is hand-written without AI assist..even without spelling check...
OOP
Get back to OOP, and this is where we handle objects' behaviours and inheritance relationships, which is being emphasized a lot in school. Not saying this is bad, or is there a better choice, OOP is good in some aspects, such as encapsulating a graphics api (especially when it comes to VULKAN), but these features are not "free". Too much abstraction makes you feel good about it, and even forget the memory usage, layout, fragmentation, heap or stack, kernel operations...In engineering world, we consider trade-offs. When it comes to a performace-sensitive case, for example, game engine and trading infrastructure, you won't want to pay anything for abstraction.
DOD (Data-Oriented-Design)
The key idea of DOD is that, a computer program is only transforming data. This is true even if it comes to a video game. Gaining exp is just an increment on your experience field, making damage is just a decrease of ennemies' HP...It's all around data, so you, the programmer should also treat everything as data, not objects. When we are doing DOD, we consider the best way to place and access the data.
AoS vs SoA
SoA is one product of DOD. This doesn't mean DOD includes SoA, cache-aware stuff...These techniques are just the results when we doing proramming towards data. Considering memory layout, you will find AoS might be filled in with lots of cold data. For example, when modeling an order, your algorithm will heavily access its price, side and quantities, but it won't care which source it came from, or who the people behind it is, and given that these orders will be placed into an array, the price field between two orders might have a big gap, aka. offset, which would give you memory fragmentaion and cache line might be angry about this, given computer fetches data in a sequence once a time into their L1 cache.
struct Order{
Time timestamp;
float price;
uint32_t quantity;
Side side;
std::string the_guy_behind_it;
std::string that_guy_s_email;
OtherStuff other_stuff;
};
auto orders = std::array<Order, MAX_ORDERS>();
for (auto order: orders) do_some_stuff_with(orders.price); // Cache Line: ?
But when with SoA, we make sure that prices are getting along with prices, and quanities are getting along with quantities, so when you want to ask for prices you get the prices into your cache line without any waste.
struct Order{
Time[MAX_ORDERS] timestamps;
float[MAX_ORDERS] prices;
....
};
...
for (auto price:order.prices) do_stuff_again(price); // All fit in your cache line!
You should also consider your hardware though! Run fastfetch to see how much cache line you get.
Database Tables
An advanced option would be treating these data as splitted table, with foreign keys to join them. This would make your program much more scalable, than ever!
Data is first tier
Your program is not designed for "general case", or you are not a good project manager! You should know what kind of data will the program be working around and then do tweaks around these data! For example, new orders are not uniformly fitted into the order book. The bids and asks are usually centerd around a point. And you the programmer should focus on such pattern and doing profilings based on it.
Avoid side effects on your functions
From the perspective of DOD, functions define the way to transform the data, so it's still around data. So it should be:
- fast
- paying nothing for abstarction
- better be parallable without locks! Anything that breaks these points is named Side Effects. Examples: Side effected functions, not thread-safe!
struct Object{
Data data;
Data data2;
...
void modify_these_data();
};
And this is Parallable! Given that the opertaion function doesn't modify any outer variable, which would be thread-safe naturally.
struct Object{
Data data;
Data data2;
...
};
// Parallel
Result operation(Object& obj);
// Sequential
ApplyResult(Object& obj, Result& result);