Where to find a source of historical futures data?
Here are some extensive sources of free ASCII data that we found reliable over time:
- Quandl, free historical futures/stocks data files. Individual and continuous contracts. Wealth-Lab provider available! (click)
- Forexite.com: Free 1-minute Forex data. Wealth-Lab provider available! (click)
- GAIN Capital, a huge source of free historical Forex data for many currency pairs (zipped, tick-based).
- Stooq, free historical daily and intraday data for U.S. and world futures, stocks, indices and macroeconomy items. Available in ASCII and MetaStock formats.
- FXHistoricalData.com, free hourly/daily Forex data
- SlickCharts, free historical Emini futures tick data files
- HistData, free historical Forex tick data files
Your suggestions are welcome.Note:
Click to download a set of Symbol Info Manager settings for TradingBlox futures data
(for the ActiveTrader standard futures portfolio) and for PremiumData futures data
. Unzip to Wealth-Lab's Data folder (see the Wealth-Lab 6 User Guide > Data on where the data are stored
; enable hidden file visibility in Windows).
How do I get a complete list of NYSE, Nasdaq, and AMEX symbols/tickers?
They are built into Wealth-Lab's Yahoo! provider > "Create New DataSet" > Classification Groups.
Whenever you need detailed information on tickers, contract specifications, etc., start by searching the official site of market where the product trades. www.nasdaq.com conveniently provides symbols downloads for all three stock markets. Just click here
and look for "Download Security List". Is it possible to edit stored data in Wealth-Lab 6?
Yes. Double-click on a bar to edit it.
Note that there also is a programmatic way that involves calling the Bars.SaveToFile
method after modifying data in your code.Streaming data loads slow.
If you use Wealth-Lab to watch streaming Intraday charts, take time to build your static intraday DataSets first. Version 6 uses the static data as backfill for streaming charts to avoid unnecessarily requesting the same data time and again.
Be mindful of the "All Data" setting when streaming. Make sure that you do not select "All Data" when requesting Streaming charts. Doing so will result in long delays as all-available intraday data is loaded for the symbol. Instead, use a specified number of fixed bars, for example. Static data loads slow.
Can I relocate the whole "Data" folder (or only some subfolder) of my Wealth-Lab 6 from C: to a different drive or directory?
- It's strongly advised to turn off On demand data update feature and keep your DataSets up to date using the Data Manager
- When using Yahoo data, turn off "Always return the partial bar" unless you're using it for live charting/trading
- If you use ASCII data, make sure to use the Cache option
There's no built-in option but it's possible by using "symlinks":
Configure Windows Explorer's Tools > Folder Options > View dialog to Show hidden files/folders
. The path of the Data folder is:
C:\Users\Windows User Name
Make sure you're linking to AppData\Roaming\...\Data folder of the same user who installed the application.Is it necessary to have access to intra-bar tick data to daytrade with Wealth-Lab?
Absolutely not. Not only tick data is more noise than information, tick charts really aren't practical for Wealth-Lab as your strategy has to execute for each tick added to a chart. Furthermore, access to intra-bar ticks is not required to trade intraday. Here's how the process is in Wealth-Lab:
How to delete (e.g. broken) data files?
- A complete bar updates (can be 1-minute, 1-day, etc.)
- Your strategy executes over all the bars in the chart and determines if it wants to trade on the next bar.
- If it doesn't, you don't do anything.
- If it does, you place orders for the next bar and they're worked by your broker. (Process complete).
There are several ways to physically delete/refresh static data:
Today is a U.S. holiday but I can't update the data for non-U.S. symbols, why?
- If data provider is supported by Data Tool, try this most convenient option. Highlight a DataSet, choose "Remove all data", then simply update your DataSet.
- If not supported by Data Tool and there's a small number of affected symbols, right click on the chart and choose the "Reload chart history" option.
- Delete the file(s) from the \Data\[Some Provider]\.. folder.
- Delete the symbol(s) from all DataSets (of a provider), and then in the Data Manager > Update Data, enable the option to "Delete data for Symbols that are not contained in DataSets".
- The Fidelity provider also has an option to delete quarantined symbols (applies to: Wealth-Lab Pro).
Most likely, because your provider supports the Market Manager
tool and you either haven't added your non-U.S. (e.g. European) symbols to a separate market, or the "U.S. Equities" market is the default market in the Market Manager. Uncheck "U.S. Equities" or add your symbols to a new market where the last date is not defined as a holiday.What's the difference between Wealth-Lab two Data folders?
There are two folders named Data (for their exact location, see Wealth-Lab User Guide > Data > Where Data are Stored
When compressed into a higher intraday scale, the bars are being built in an unexpected manner.
- In Wealth-Lab's installation directory under Program Files. This one's a part of the installation, copied for new Windows users that use Wealth-Lab on the same machine, and is used to "initialize" new installations with built-in DataSets, Strategies, Rules etc.
- Under user's AppData folder. This is the storage of your own Strategies, program/extension settings, downloaded data etc.
- For example, you'd expect the first four 15-minute bars to be compressed into a single 60 min one. However, Wealth-Lab keeps the first one and start merging at the second bar.
Wealth-Lab always assumes end-of-bar labeling. Consequently, if your first bar is labeled 09:30, then it is assumed that this bar is a result of trading between 09:15 and 09:30, and, unless specified otherwise by the data provider, 09:30 is the market opening time, which marks the hourly divisions. Consequently, the first four bars to be compressed into the first hourly bar must be time stamped as 09:45, 10:00, 10:15, and finally 10:30. You can change that by adjusting the market's opening time by assigning a market group to the symbol with the proper market open time in the Market Manager tool
.Is it possible to create a DataSet from bars generated using SaveToFile (*.WL)?
Yes. Follow this guide
.How to append data from one data source to another?
See post #2 in this thread
Data providersIs there any API for the development of a data adapter?
Yes, Wealth-Lab Pro/Developer 6 contains a number of APIs to build your own streaming and static data providers as well as many other extensions (see Wealth-Lab Version 6 (.NET) Development Guide
At the moment these documents are not available but our customers are welcome to enter a new support ticket in our support portal and/or take a look at the source code of some of MS123 Providers
for a working example of a static and streaming data providers.
Are there any 3rd party data providers for use with Wealth-Lab 6? What about real-time Interactive Brokers data?
Fundamental DataI cannot find any fundamental data in WLD 6.
One fundamental data provider is included in the installation package - part of the MSN Static, Streaming and Fundamental suite. Registered customers can install these free Extensions:
Can I install fundamental data providers by 3rd party sites?
If your data vendor provides fundamental items not found in MSN and YCharts and it has some kind of API, motivated developers might be able to build a fundamental provider. Note:
Disclosing the fundamental provider API is not planned.
I dragged and dropped a fundamental item on the chart but it's not shown.
Fundamental data has to be updated before using it in charting and/or backtesting, and to do that, activate desired fundamental provider(s) in the "Fundamental Data Providers" list on the bottom of Data Manager's "Update Data" tab. Don't forget to reopen the chart after applying changes.
Google data providerThe provider does not return any data (0 bars) for some symbols. Why?
If Google Finance website for a given symbol does not have a widget called "Export" with "Download to spreadsheet" option, then the historical data for this symbol can not be downloaded from Google Finance. For example:
- Symbol with downlodable data: AAPL
- Symbol whose historical data can NOT be downloaded: VXV
Before raising a support issue, you should visit the Google Finance website and make sure that:
- the data there is actual
- the historical data for a symbol can be downloaded
Yahoo! DataIs it possible to get the data for futures or Forex from Yahoo?
No, because Yahoo! does not provide historical futures data. It's very easy to check whether a Y! symbol has historical data or not. When you see that the side menu for a symbol contains the "Historical Prices" link, you know that this symbol is enabled. For example, check out the historical data lookup page for S&P December 2016 futures
or Forex GBP/USD
.Is it possible to get intraday historical stock using the Yahoo provider?
No, because Yahoo! does not provide historical intraday stock data required for backfill.
ASCII DataI'm unable to load intraday ASCII data. Time format is Hmm.
Update the ASCII Provider Extension to version 1.3 or higher. (For more information, see Errors > Data
).How to copy format from DataSet (ASCII provider)?Symptoms:
There is an option to "Copy format from DataSet" in the ASCII Provider, but it is not highlighted for some reason. Solution:
The option "Copy Format from a DataSet" is helpful when you have a repetitive task of creating related ASCII datasets. If they share the same format, you don't have to specify its details for DataSet #2, #3 and so on. If at least one ASCII DataSet exists in WL6, the option becomes highlighted for you to replicate saved settings (and save time).
How do I import an unusual ASCII data file with multiple separators?
When your data looks something like this:
44.3400001525879,44.3699989318848,44.3400001525879,44.3600006103516,85675,"3/27/2008 12:08:00 PM"
It means there's more than one field separator at work. It's not typical for data providers but the ASCII data adapter (version 1.2.3 and higher) has means to handle this sort of inconsistency. To indicate that the source data file contains multiple separators, specify them one after another with the plus (+
) or ampersand (&
) sign in the middle. For example, to set up the following data you would enter ,+"
in the Field Separator field:
44.3400001525879,44.3699989318848,44.3400001525879,44.3600006103516,85675,"3/27/2008 12:08:00 PM"
44.3698997497559,44.3699989318848,44.3600006103516,44.3600006103516,47826,"3/27/2008 12:09:00 PM"
Metastock DataI can't find the MetaStock format in "Create a new DataSet". My MetaStock DataSet is lost and won't appear.
You're using 64-bit Wealth-Lab 6.4 (or earlier build). Before version 6.5, the MetaStock provider wasn't 64-bit compatible so the ability to create and
read from MetaStock DataSet existed only in 32-bit installations. Solution:
upgrade to Wealth-Lab 6.5.
for versions 6.4 and below: install WL6 32-bit alongside WL6 64-bit to read MetaStock datasources on your 64-bit Windows.
: applies to Wealth-Lab Pro
In general, please call Fidelity directly about data issues and data requests.
I have a problem with Fidelity data.
We (MS123 LLC, a 3rd party) are not Fidelity. Data is not our business. We can not help with Fidelity data. Please call Fidelity when you have a problem with Fidelity data.Can I programmatically access GICS data (i.e. stock classification by sector and industry)?
Yes. This functionality was added in WLP Version 5.5 and is documented in the WealthScript Programming Guide (see Fundamental Analysis - GICS Data
). A few assorted helpful links:
How can I get TTM, MRQ and FTM values of estimated earnings?
See these discussions for the insight and code examples:Can I have access to premarket or aftermarket data in strategy?
No. Fidelity does not supply pre/post data to Wealth-Lab. You may use ActiveTrader Pro to view it or, as a workaround, try to get this data with a page scrape from external web site like Yahoo in your strategy using C#.There is a DataSet called "Market Sentiment" (Fidelity). What are these tickers? Can they be traded?
They're market sentiment indicators and are described in the WealthScript Programming Guide, Fundamental Analysis > Market Sentiment Data
. They aren't tradeables. The minimum time frame is daily
. They simply indicate market sentiment and breadth: the number of advancing/declining/unchanged/total securities (with their volume) and news highs/lows made for each major exchange. More details available at the sites like Investopedia
.Fidelity data does not match the same symbol in Active Trader Pro.Symptoms:
Assuming the same data scale, static intraday Data by Fidelity for the same bar is different from its corresponding Active Trader Pro (ATP) reading.Reason:
In previous versions of ATP, there was a difference in intraday time stamps between it and WL Pro. The bar was timestamped at the start of it in ATP and WLP marks the same bar with the time at the end of bar. Newer versions (e.g. 10.6.766) label bars by its close time like WLP. Solution: upgrade ATP.Symptoms:
The intraday close and the daily close did not match in Fidelity data.Reason:
See this forum topic
for an in-depth discussion of the phenomenon.