Speeding Bloomberg Intraday Bar Downloads with Parallel Requests
Summary
The discussion addresses slow downloads of intraday bar data through the Bloomberg API. The accepted response recommends issuing multiple requests concurrently rather than relying on a single request loop, while assigning each request a distinct identifier. Response handling can then use the identifier to associate incoming messages with the correct request. The example describes starting worker threads, waiting for their completion, and collecting the responses.
The author reports that parallelizing a batch of requests reduced the overall elapsed time, even though it did not make any individual request faster. The post provides implementation guidance for coordinating requests, but no measured timings, concurrency limits, or comparison across different data volumes. Its result is an account of one user's experience, and practical gains may depend on API constraints, connection setup, response processing, and the amount of data requested. The discussion concerns data retrieval efficiency rather than a trading signal or strategy.
Key ideas
- Concurrent API requests can reduce total time when downloading multiple intraday data batches.
- Distinct request identifiers help associate responses with the requests that generated them.
- A thread-safe request procedure and coordinated response handling are needed for parallel downloads.
- The reported speedup applies to the batch's elapsed time, not necessarily to each request.
- The discussion gives no timings or guidance on Bloomberg API concurrency limits.
Tags
Full text
# How to download efficiently intraday data with Bloomberg API?
# How to download efficiently intraday data with Bloomberg API?
I'm downloading intraday bar data using Bloomberg API and C#.
I have adapted the official Bloomberg c# "IntradayBarExample” to suit my needs.
However downloads are really slow, I found this post recommending to use a dedicated `EventQueue` instead of calling `NextEvent()` on a `Session` object. But the improvement is not really detectable.
As illustration, you can download the 5Min Bar – Best Bid Event- of the last 100 days for AAL UW Equity ticker, it takes very long time to obtain at the end not so much data (< 1Mo).
Did you face the same problem ?
## Answer by Malick (score 0, accepted)
https://quant.stackexchange.com/a/23009
As suggested by assylias I have modified my code to run requests in several threads. In case you need to do so, these are some valuable informations :
1- Create a thread-safe request procedure and assign differents `requestID` to your requests. Pass these arguments to `processResponseEvent` via the `eventloop` function. This will allow you to make a check regarding the message you are receiving by using a simple condition:
```
if (msg.CorrelationID != requestID)
{
System.Console.WriteLine("WRONG ID ");
return;
}
```
2- Make your request in parallel, meaning you enclosed the aforementioned request in a loop such as :
```
WaitHandle[] waitHandles = new WaitHandle[numOfThreads];
for (int i = 0; i < numOfThreads; i++)
{
var j = i;
var handle = new EventWaitHandle(false, EventResetMode.ManualReset);
var thread = new Thread(() =>
{
/// CALL HERE YOU REQUEST PROCEDURE FROM STEP 1
handle.Set();
});
waitHandles[j] = handle;
thread.Start();
}
foreach (var ee in waitHandles)
ee.WaitOne();
```
At the end it won't increase the waiting time for each request but since your are performing a bunch of them together you'll still save a LOT of time.
Ps: you can use the variable `i` (from `for (int i = 0; i < numOfThreads; i++)`) as `requestID`.Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.