FAQ | Data and Data Providers

Modified on 2021/03/13 12:17 by Eugene — Categorized as: Knowledge Base

Data



Where to find a source of historical futures data?

Here are some extensive sources of free ASCII data that we found reliable over time:


Your suggestions are welcome.

Note: Click to download a set of Symbol Info Manager settings for TradingBlox futures data (for the ActiveTrader standard futures portfolio) and for PremiumData futures data. Unzip to Wealth-Lab's Data folder (see the Wealth-Lab 6 User Guide > Data on where the data are stored; enable hidden file visibility in Windows).

How do I get a complete list of NYSE, Nasdaq, and AMEX symbols/tickers?

They are built into Wealth-Lab's Yahoo! provider > "Create New DataSet" > Classification Groups.

Whenever you need detailed information on tickers, contract specifications, etc., start by searching the official site of market where the product trades.

Is it possible to edit stored data in Wealth-Lab 6?

Yes. Double-click on a bar to edit it.

Note that there also is a programmatic way that involves calling the Bars.SaveToFile method after modifying data in your code.

Streaming data loads slow.

If you use Wealth-Lab to watch streaming Intraday charts, take time to build your static intraday DataSets first. Version 6 uses the static data as backfill for streaming charts to avoid unnecessarily requesting the same data time and again.

Be mindful of the "All Data" setting when streaming. Make sure that you do not select "All Data" when requesting Streaming charts. Doing so will result in long delays as all-available intraday data is loaded for the symbol. Instead, use a specified number of fixed bars, for example.

Static data loads slow.

Suggestions:


Can I relocate the whole "Data" folder (or only some subfolder) of my Wealth-Lab 6 from C: to a different drive or directory?

There's no built-in option but it's possible by using "symlinks":


Configure Windows Explorer's Tools > Folder Options > View dialog to Show hidden files/folders. The path of the Data folder is:

C:\Users\Windows User Name\AppData\Roaming\Fidelity Investments\WealthLabDev\1.0.0.0\Data

Make sure you're linking to AppData\Roaming\...\Data folder of the same user who installed the application.

Is it necessary to have access to intra-bar tick data to daytrade with Wealth-Lab?

Absolutely not. Not only tick data is more noise than information, tick charts really aren't practical for Wealth-Lab as your strategy has to execute for each tick added to a chart. Furthermore, access to intra-bar ticks is not required to trade intraday. Here's how the process is in Wealth-Lab:

  1. A complete bar updates (can be 1-minute, 1-day, etc.)
  2. Your strategy executes over all the bars in the chart and determines if it wants to trade on the next bar.
    • If it doesn't, you don't do anything.
    • If it does, you place orders for the next bar and they're worked by your broker. (Process complete).

How to delete (e.g. broken) data files?

There are several ways to physically delete/refresh static data:

  1. If data provider is supported by Data Tool, try this most convenient option. Highlight a DataSet, choose "Remove all data", then simply update your DataSet.
  2. If not supported by Data Tool and there's a small number of affected symbols, right click on the chart and choose the "Reload chart history" option.
  3. Delete the file(s) from the \Data\[Some Provider]\.. folder.
  4. Delete the symbol(s) from all DataSets (of a provider), and then in the Data Manager > Update Data, enable the option to "Delete data for Symbols that are not contained in DataSets".

Today is a U.S. holiday but I can't update the data for non-U.S. symbols, why?

Most likely, because your provider supports the Market Manager tool and you either haven't added your non-U.S. (e.g. European) symbols to a separate market, or the "U.S. Equities" market is the default market in the Market Manager. Uncheck "U.S. Equities" or add your symbols to a new market where the last date is not defined as a holiday.

What's the difference between Wealth-Lab two Data folders?

There are two folders named Data (for their exact location, see Wealth-Lab User Guide > Data > Where Data are Stored):

  1. In Wealth-Lab's installation directory under Program Files. This one's a part of the installation, copied for new Windows users that use Wealth-Lab on the same machine, and is used to "initialize" new installations with built-in DataSets, Strategies, Rules etc.
  2. Under user's AppData folder. This is the storage of your own Strategies, program/extension settings, downloaded data etc.

When compressed into a higher intraday scale, the bars are being built in an unexpected manner.


Wealth-Lab always assumes end-of-bar labeling. Consequently, if your first bar is labeled 09:30, then it is assumed that this bar is a result of trading between 09:15 and 09:30, and, unless specified otherwise by the data provider, 09:30 is the market opening time, which marks the hourly divisions. Consequently, the first four bars to be compressed into the first hourly bar must be time stamped as 09:45, 10:00, 10:15, and finally 10:30. You can change that by adjusting the market's opening time by assigning a market group to the symbol with the proper market open time in the Market Manager tool.

Is it possible to create a DataSet from bars generated using SaveToFile (*.WL)?

Yes. Follow this guide.

How to append data from one data source to another?

See post #2 in this thread.



Data providers



Is there any API for the development of a data adapter?

Yes, Wealth-Lab Pro/Developer 6 contains a number of APIs to build your own streaming and static data providers as well as many other extensions (see Wealth-Lab Version 6 (.NET) Development Guide). Also, our customers can take a look at the source code of some of MS123 Providers for a working example of a static and streaming data providers.

Are there any 3rd party data providers for use with Wealth-Lab 6? What about Interactive Brokers data?

Yes, you can install them from Extensions section. A 3rd party has created a turnkey automated trading solution for Interactive Brokers that includes data providers.

Fundamental Data



I cannot find any fundamental data in WLD 6.

One fundamental data provider is included in the installation package: Yahoo Static, Streaming and Fundamental suite (contains dividend and split data). Registered customers may choose from a wide choice of free Extensions to install: a collection of 11 fundamental, economic and news providers, YCharts, Nasdaq, Morningstar, Zacks and more.

I dragged and dropped a fundamental item on the chart but it's not shown.

Fundamental data has to be updated before using it in charting and/or backtesting, and to do that, activate desired fundamental provider(s) in the "Fundamental Data Providers" list on the bottom of Data Manager's "Update Data" tab. Don't forget to reopen the chart after applying changes.



Yahoo! Data



Is it possible to get the data for futures or Forex from Yahoo?

Yes, v2017.12 supports static and streaming data for continuous Futures contracts (e.g. Crude Oil) and Forex currencies (e.g. EURUSD=X.

Is it possible to get intraday historical stock using the Yahoo provider?

No, because Yahoo! does not provide historical intraday stock data required for backfill.

ASCII Data



How to copy format from DataSet (ASCII provider)?

Symptoms:
There is an option to "Copy format from DataSet" in the ASCII Provider, but it is not highlighted for some reason.

Solution:
The option "Copy Format from a DataSet" is helpful when you have a repetitive task of creating related ASCII datasets. If they share the same format, you don't have to specify its details for DataSet #2, #3 and so on. If at least one ASCII DataSet exists in WL6, the option becomes highlighted for you to replicate saved settings (and save time).

How do I import an unusual ASCII data file with multiple separators?

When your data looks something like this:

"AUD/JPY","87.880000","87.290000","2006-07-17 16:59:11" 44.3400001525879,44.3699989318848,44.3400001525879,44.3600006103516,85675,"3/27/2008 12:08:00 PM"

It means there's more than one field separator at work. It's not typical for data providers but the ASCII data adapter (version 1.2.3 and higher) has means to handle this sort of inconsistency. To indicate that the source data file contains multiple separators, specify them one after another with the plus (+) or ampersand (&) sign in the middle. For example, to set up the following data you would enter ,+" in the Field Separator field:

44.3400001525879,44.3699989318848,44.3400001525879,44.3600006103516,85675,"3/27/2008 12:08:00 PM" 44.3698997497559,44.3699989318848,44.3600006103516,44.3600006103516,47826,"3/27/2008 12:09:00 PM"