Wealth-Lab Wiki

API ChartStyles Community Components Community Indicators IndexDefinitions Knowledge Base Misc Optimizers Pending Deletion PosSizers Providers Standard Indicators TASC Traders Tips TASCIndicators Tutorial Videos Visualizers
RSS

Navigation


Quick Search
»
Advanced Search »


Quandl provider suite

RSS

What it is

This is a suite comprised of static and fundamental providers that download daily (EOD) historical data from Quandl website.

  • free and subscription-based static EOD data for global instruments (futures, commodities, rates, stocks, indices, bonds, currencies...)
  • free COT data (in legacy and new formats)
  • free sentiment data by AAII
  • subscription-based fundamental data by Zacks (up to 200 fundamental items, dividends, earnings esimates/surprises, analyst ratings)

More instruments can be supported. Please suggest your candidates for inclusion in this forum thread:


Setting up

It's a must that before using the provider you obtain a so called Auth Token from Quandl.com. To obtain it, register on their website, activate your account and copy the sequence from Account Settings section of the website. To finish setup, either type it in the Data Manager's Quandl tab, or create a new static DataSet:

Enter Quandl auth token in Data Manager

Enter Quandl auth token in Data Manager


Note! Without completing this step, you will be able to make only 50 data requests per day.

Finding symbols

Finding symbol tickers at Quandl is easy. A Quandl ticker always consists of two strings separated with slash: data collection name and data item name. Here's how it works.

For example, you wanted to get access to Crude Oil futures continuous contract and found it by the following link:

https://www.quandl.com/OFDP/FUTURE_B1-ICE-Brent-Crude-Oil-Futures-Continuous-Contract-1-B1-Front-Month

The highlighted part, OFDP/FUTURE_B1, is the unique ticker that you enter into Wealth-Lab. It always ends before the first "-" symbol in the URL.

Image

How it works

Unlike most other data providers, it's important to understand how Quandl provider works. Quandl is a large data community where anyone can upload their data. Consequently, the data exists in a multitude of formats. The data is so far from being uniform that groups of related instruments (let's say futures) might have different field sets! The data for the same instrument can be paid and free. Free data can have defects (omissions, missing components for a data range etc.) This creates big problem for software developers who would like to support Quandl data. For example, in some software packages you have to manually configure each instrument through a configuration file. This is not the Wealth-Lab way.

Our solution is dynamic parsing. Wealth-Lab does all the low level job of recognizing the data format behind the scenes. It will recognize whether the data is simple OHLC for a stock, or it's a futures contract with special fields like Open Interest, or it's a Close-only currency, or sentiment data like Commitments of Traders, or some spot instrument or a rate. They've been uploaded by different vendors but you don't have to configure anything.

Like we already said, the free data can sometimes be quite dirty: for example, it may be missing certain fields for several months and then they reappear. In these cases, the provider has to make assumptions about the data and replace the missing Open, High, and/or Low fields with Close.

COT data

Among the many types of data, supported is Commitments of Traders data which can be downloaded in both legacy (e.g. CFTC/T_F_ALL) and new formats (e.g. CFTC/T_F_L_ALL) as well as open interest.

  • Supported named series for symbols in new CFTC format:

Open Interest, Producer/Merchant/Processor/User Longs, Producer/Merchant/Processor/User Shorts, Swap Dealer Longs, Swap Dealer Shorts, Swap Dealer Spreads, Money Manager Longs, Money Manager Shorts, Money Manager Spreads, Other Reportable Longs, Other Reportable Shorts, Other Reportable Spreads, Total Reportable Longs, Total Reportable Shorts, Non Reportable Longs, Non Reportable Shorts

  • Supported named series for symbols in legacy CFTC format:

Open Interest, Noncommercial Long, Noncommercial Short, Noncommercial Spreads, Commercial Long, Commercial Short, Total Long Total Shorts, Nonreportable Positions Long, Nonreportable Positions Short

Here's an example of plotting them all at once:

using System;
using System.Collections.Generic;
using System.Text;
using System.Drawing;
using WealthLab;
using WealthLab.Indicators;

namespace WealthLab.Strategies { public class MyStrategy : WealthScript { Random r = new Random(); private Color RandomColor() { Color randomColor = Color.FromArgb(r.Next(255), r.Next(255), r.Next(255)); return randomColor; } protected override void Execute() { if( Bars.HasNamedDataSeries ) { ChartPane p = CreatePane( 100, true, true ); foreach( DataSeries ds in Bars.NamedSeries ) PlotSeries( p, ds, RandomColor(), LineStyle.Solid, 1 ); } else PrintDebug( Bars.Symbol + " does not contain named series" ); } } }

CoT data by Quandl

CoT data by Quandl


This example illustrates creation of a custom DataSeries from the COT Named Series data defined as Commercial Long - Commercial Short and then applying RSI indicator to the custom DataSeries to normalize the custom index. It can only be used with Quandl symbols containing legacy series i.e. ending with "_L_ALL":


using System;
using System.Collections.Generic;
using System.Text;
using System.Drawing;
using WealthLab;
using WealthLab.Indicators;

namespace WealthLab.Strategies { public class MyStrategy : WealthScript { protected override void Execute() { // For use on symbols containing legacy series i.e. symbols ending with "_L_ALL" if( Bars.HasNamedDataSeries ) { DataSeries cl = Bars.FindNamedSeries("Commercial Long"); DataSeries cs = Bars.FindNamedSeries("Commercial Short"); DataSeries diff = cl - cs; RSI rsiDiff = RSI.Series( diff, 2 ); rsiDiff.Description = "RSI on Difference between Commercial Long/Short"; if( cl != null && cs != null ) { ChartPane p2 = CreatePane( 25, true, true ); ChartPane p1 = CreatePane( 25, true, true ); ChartPane p = CreatePane( 25, true, true );

PlotSeries( p, cl, Color.Green, LineStyle.Solid, 1 ); PlotSeries( p, cs, Color.Red, LineStyle.Solid, 1 ); PlotSeries( p1, diff, Color.Blue, LineStyle.Histogram, 2 ); PlotSeriesOscillator( p2, rsiDiff, 70, 30, Color.FromArgb(30,Color.Red), Color.FromArgb(30,Color.Blue), Color.DarkMagenta, LineStyle.Solid, 1 ); } } else PrintDebug( Bars.Symbol + " does not contain named series" ); } } }

AAII data

American Association of Individual Investors (AAII) publishes a weekly survey of investor sentiment since 1987. The AAII Investor Sentiment Survey measures the percentage of individual investors who are bullish, bearish, and neutral on the stock market for the next six months. Provider supports this data as symbol AAII/AAII_SENTIMENT and exposes poll results as Named DataSeries - much like COT data:

  • Bullish, Bearish, Neutral, Bull-Bear Spread

AAII data by Quandl

AAII data by Quandl


This example illustrates how to plot the AAII named DataSeries. It can only be used with the Quandl symbol AAII/AAII_SENTIMENT:

using System;
using System.Collections.Generic;
using System.Text;
using System.Drawing;
using WealthLab;
using WealthLab.Indicators;

namespace WealthLab.Strategies { public class MyStrategy : WealthScript { protected override void Execute() { // For use on symbol AAII/AAII_SENTIMENT only if( Bars.HasNamedDataSeries ) { DataSeries bu = Bars.FindNamedSeries("Bullish"); DataSeries be = Bars.FindNamedSeries("Bearish"); DataSeries n = Bars.FindNamedSeries("Neutral"); DataSeries s = Bars.FindNamedSeries("Bull-Bear Spread"); if(bu != null && be != null && n != null && s != null) { ChartPane p1 = CreatePane( int.MaxValue, true, true ); ChartPane p = CreatePane( int.MaxValue, true, true );

PlotSeries( p, bu, Color.Green, LineStyle.Solid, 2 ); PlotSeries( p, be, Color.Red, LineStyle.Solid, 2 ); PlotSeries( p, n, Color.DarkBlue, LineStyle.Solid, 1 ); PlotSeries( p1, s, Color.Blue, LineStyle.Histogram, 2 ); } } else PrintDebug( Bars.Symbol + " does not contain named series" ); } } }



Bitcoin data

If interested in Bitcoin, the symbols for BTC/USD are: BCHARTS/BITSTAMPUSD and BCHAIN/MKPRU. There are some supporting time series that the provider can recognize like number of Bitcoin users or exchange trade volume which are available in datasets like Bitcoin Charts Exchange Rate Data and Blockchain.

Issues and limitations

  • Provider does not have the means to adjust for splits and dividends. To maintain your data in good shape, either perform splits manually or reload the DataSet from scratch using the Data Tool's "Remove all data" feature and then updating the DataSet.

  • While provider appends security names to symbols (starting from v2017.11), it cannot do so for the symbols in WIKI/ or EOD/ groups.

  • Quandl has a fair usage policy and will throttle your data update if the provider makes more than 2000 calls in a 10 minute slot or if Quandl thinks it's "speeding". If you experience that part of a big DataSet hasn't been updated or a supported symbol suddenly returns "No data", try this:
    1. When doing an initial backfill, break up your big DataSet in parts and update them manually.
    2. If updating a big DataSet starts returning 0 bars, take a break and repeat shortly.

Important Disclaimer: The information provided by Wealth-Lab is strictly for informational purposes and is not to be construed as advice or solicitation to buy or sell any security.  The owner of Wealth-Lab.com assumes no liability resulting from the use of the material contained herein for investment purposes. By using this web site, you agree to the terms of this disclaimer and our Terms of Use.

Used under license from FMR Corp. Copyright 2008 FMR Corp. All rights reserved.


ScrewTurn Wiki. Some of the icons created by FamFamFam.