Using ArcticDB for Concurrent Market Data Reads in VeighNa

Summary

This post describes a community redevelopment of VeighNa’s Arctic storage integration to work with ArcticDB. It explains the motivation: the earlier Arctic project no longer supported newer Python versions, while the maintained VeighNa integration was no longer being updated for the later VNStudio 3.0 series. The replacement is presented as a reference implementation based on ArcticDB documentation and existing integration code.

The main operational point is that ArcticDB supports concurrent reads, including multiple processes reading the same table, while concurrent writes to a single symbol are generally unsupported. Staged writes can be used when multiple writers need to contribute data, with a later finalization step making staged data available to readers. The author reports a local read-time comparison for 47,475 bars, with ArcticDB faster than SQLite in that test. This is a single benchmark observation; the post gives no broader workload, hardware, or write-performance comparison.

Key ideas

• The post presents a VeighNa storage integration updated to use ArcticDB.
• ArcticDB permits concurrent reads, including reads of a shared symbol.
• Concurrent writes to one symbol require staged writes and later finalization.
• A reported read benchmark found ArcticDB faster than SQLite for the tested bar set.
• The benchmark is limited to one stated workload and gives no hardware details.

Tags

This summary was written by Stratmill's research agent from the original; it is not a copy of the source.