Separating Brokerage Data Collection from a Web Application
Summary
The document considers whether Interactive Brokers data can be fetched directly by a hosted web application when its API requires Trader Workstation or IB Gateway to run as a separate executable. The responses argue that continuously collecting live or tick data is generally better handled by a dedicated process on a machine or server that permits long-running programs. A standard web hosting environment may stop such processes, and combining data collection with the web application can make the service less reliable when a background task fails.
The suggested architecture separates collection and processing from presentation: a persistent process fetches and prepares the data, then a web server makes it available to users. The advice is aimed at live data collection and is not a comprehensive survey of brokerage APIs. The provider names and comments about requiring a graphical environment reflect the original answers and may be outdated; the document does not establish current Interactive Brokers authentication options or hosting requirements. It gives no comparative evaluation of alternative brokers.
Key ideas
- The responses recommend running persistent market-data collection outside a conventional web request process.
- A dedicated machine or server can host the brokerage software and long-running data collector.
- Separating data capture and processing from the web interface can reduce the chance that one failure disables the whole application.
- A web application can present data gathered and prepared by a separate service.
- The operational details in the responses may be dated and do not confirm current brokerage API or hosting requirements.
Tags
Full text
# Does Interactive Brokers (IB) have a Web friendly API? # Does Interactive Brokers (IB) have a Web friendly API? The requirement I am given is to implement a web ppplication which utilizes Interactive Brokers's API to fetch data. I went through the IB API web page and came across two viable methods: TWS and IB Gateway. But both method require proprietary executables to be running. This doesn't make sense from a web perspective as my hosting provider will not allow to run an executable on their infrastructure. Is it not possible to just access their API using Username/Password or some API Key or something similar from a Web Server? If it's not possible could you please share what other companies (like IB) have web friendly APIs. ## Answer by Frankie (score 2) https://quant.stackexchange.com/a/4342 Don't try to capture LIVE tick data using a WebApp. I'm not saying it can't be done, I'm just saying you would get zero benefits and you would have to work way harder to make it functional. Web servers are designed with a premise, serve the user the requested data as fast as possible and free that resource up. - You would have to fight the server logic (as it is not designed to run that way) - You would have to fight your provider (as it may interpret the server as crashed and close it) In order to capture tick-data using IB use Java and look into VPS (virtual private servers) where you are allowed to run whatever process you want. Over the last 4 years I've been using the following companies for several long-polling apps (both finance and non-finance related) with great success. - linode - slicehost && rackspace (slicehost will be absorved by rackspace) You can even run X on these headless systems (and you will need X to run Interactive Brokers API - both on the TWS version and on the Gateway one). ## Answer by Matt Wolf (score 0) https://quant.stackexchange.com/a/4368 I second parts of Frankie's answer here but for a different reason and with additional caveats: First yes, do not run a process that does not serve content you already provide as a web application. The point of a web app is to simply make content available on a standardized medium for distribution purposes and not for computational or data gathering purposes. I have seen people implementing a whole strategy testing and trading platform on a web app, which imho makes zero sense. Its a very inefficient and prone to error process. The point is that if a single of your computations goes down it essentially renders the whole web app useless. The correct way to run this is to gather your data through the API on a server or local machine (whatever you prefer) and to then offer the gathered data (raw or cleaned or otherwise processed) to users through a web application. You actually answered the question yourself, you often need other processes that run in order to gather the data you inquire and you don't want to have to mess with this on a web app and or even worse force the user of the web app to run other processes just in order to make it work. Gather the data on a machine, process the data, make the data available through a web server and use the web app as interface. Simple as that.
Shown in full with attribution under the source's licence. Licence: CC BY-SA 4.0 (Stack Exchange)
This summary was written by Stratmill's research agent from the original; it is not a copy of the source.