Quandl provider suite

Modified on 2017/12/06 10:14 by Eugene — Categorized as: Providers

What it is

This is a suite comprised of static and fundamental providers that download daily (EOD) historical data from Quandl website.


More instruments can be supported. Please suggest your candidates for inclusion in this forum thread:


Setting up

It's a must that before using the provider you obtain a so called Auth Token from Quandl.com. To obtain it, register on their website, activate your account and copy the sequence from Account Settings section of the website. To finish setup, either type it in the Data Manager's Quandl tab, or create a new static DataSet:

Enter Quandl auth token in Data Manager

Enter Quandl auth token in Data Manager


Note! Without completing this step, you will be able to make only 50 data requests per day.

Finding symbols

Finding symbol tickers at Quandl is easy. A Quandl ticker always consists of two strings separated with slash: data collection name and data item name. Here's how it works.

For example, you wanted to get access to Crude Oil futures continuous contract and found it by the following link:

https://www.quandl.com/CHRIS/ICE_B1-ICE-Brent-Crude-Oil-Futures-Continuous-Contract-1-B1-Front-Month

The highlighted part, CHRIS/ICE_B1, is the unique ticker that you enter into Wealth-Lab. It always ends before the first "-" symbol in the URL.

Image

How it works

Unlike most other data providers, it's important to understand how Quandl provider works. Quandl is a large data community where anyone can upload their data. Consequently, the data exists in a multitude of formats. The data is so far from being uniform that groups of related instruments (let's say futures) might have different field sets! The data for the same instrument can be paid and free. Free data can have defects (omissions, missing components for a data range etc.) This creates big problem for software developers who would like to support Quandl data. For example, in some software packages you have to manually configure each instrument through a configuration file. This is not the Wealth-Lab way.

Our solution is dynamic parsing. Wealth-Lab does all the low level job of recognizing the data format behind the scenes. It will recognize whether the data is simple OHLC for a stock, or it's a futures contract with special fields like Open Interest, or it's a Close-only currency, or sentiment data like Commitments of Traders, or some spot instrument or a rate. They've been uploaded by different vendors but you don't have to configure anything.

Like we already said, the free data can sometimes be quite dirty: for example, it may be missing certain fields for several months and then they reappear. In these cases, the provider has to make assumptions about the data and replace the missing Open, High, and/or Low fields with Close.

COT data

Among the many types of data, supported is Commitments of Traders data which can be downloaded in both legacy (e.g. CFTC/T_F_ALL) and new formats (e.g. CFTC/T_F_L_ALL) as well as open interest.


Open Interest, Producer/Merchant/Processor/User Longs, Producer/Merchant/Processor/User Shorts, Swap Dealer Longs, Swap Dealer Shorts, Swap Dealer Spreads, Money Manager Longs, Money Manager Shorts, Money Manager Spreads, Other Reportable Longs, Other Reportable Shorts, Other Reportable Spreads, Total Reportable Longs, Total Reportable Shorts, Non Reportable Longs, Non Reportable Shorts


Open Interest, Noncommercial Long, Noncommercial Short, Noncommercial Spreads, Commercial Long, Commercial Short, Total Long Total Shorts, Nonreportable Positions Long, Nonreportable Positions Short

Here's an example of plotting them all at once:

using System;
using System.Collections.Generic;
using System.Text;
using System.Drawing;
using WealthLab;
using WealthLab.Indicators;

namespace WealthLab.Strategies { public class MyStrategy : WealthScript { Random r = new Random(); private Color RandomColor() { Color randomColor = Color.FromArgb(r.Next(255), r.Next(255), r.Next(255)); return randomColor; } protected override void Execute() { if( Bars.HasNamedDataSeries ) { ChartPane p = CreatePane( 100, true, true ); foreach( DataSeries ds in Bars.NamedSeries ) PlotSeries( p, ds, RandomColor(), LineStyle.Solid, 1 ); } else PrintDebug( Bars.Symbol + " does not contain named series" ); } } }

CoT data by Quandl

CoT data by Quandl


This example illustrates creation of a custom DataSeries from the COT Named Series data defined as Commercial Long - Commercial Short and then applying RSI indicator to the custom DataSeries to normalize the custom index. It can only be used with Quandl symbols containing legacy series i.e. ending with "_L_ALL":


using System;
using System.Collections.Generic;
using System.Text;
using System.Drawing;
using WealthLab;
using WealthLab.Indicators;

namespace WealthLab.Strategies { public class MyStrategy : WealthScript { protected override void Execute() { // For use on symbols containing legacy series i.e. symbols ending with "_L_ALL" if( Bars.HasNamedDataSeries ) { DataSeries cl = Bars.FindNamedSeries("Commercial Long"); DataSeries cs = Bars.FindNamedSeries("Commercial Short"); DataSeries diff = cl - cs; RSI rsiDiff = RSI.Series( diff, 2 ); rsiDiff.Description = "RSI on Difference between Commercial Long/Short"; if( cl != null && cs != null ) { ChartPane p2 = CreatePane( 25, true, true ); ChartPane p1 = CreatePane( 25, true, true ); ChartPane p = CreatePane( 25, true, true );

PlotSeries( p, cl, Color.Green, LineStyle.Solid, 1 ); PlotSeries( p, cs, Color.Red, LineStyle.Solid, 1 ); PlotSeries( p1, diff, Color.Blue, LineStyle.Histogram, 2 ); PlotSeriesOscillator( p2, rsiDiff, 70, 30, Color.FromArgb(30,Color.Red), Color.FromArgb(30,Color.Blue), Color.DarkMagenta, LineStyle.Solid, 1 ); } } else PrintDebug( Bars.Symbol + " does not contain named series" ); } } }

AAII data

American Association of Individual Investors (AAII) publishes a weekly survey of investor sentiment since 1987. The AAII Investor Sentiment Survey measures the percentage of individual investors who are bullish, bearish, and neutral on the stock market for the next six months. Provider supports this data as symbol AAII/AAII_SENTIMENT and exposes poll results as Named DataSeries - much like COT data:


AAII data by Quandl

AAII data by Quandl


This example illustrates how to plot the AAII named DataSeries. It can only be used with the Quandl symbol AAII/AAII_SENTIMENT:

if( Bars.HasNamedDataSeries && Bars.Symbol == "AAII/AAII_SENTIMENT" )
{
	DataSeries bu = Bars.FindNamedSeries("Bullish");
	DataSeries be = Bars.FindNamedSeries("Bearish");
	DataSeries n = Bars.FindNamedSeries("Neutral");
	DataSeries s = Bars.FindNamedSeries("Bull-Bear Spread");

if(bu != null && be != null && n != null && s != null) { ChartPane p1 = CreatePane( int.MaxValue, true, true ); ChartPane p = CreatePane( int.MaxValue, true, true );

PlotSeries( p, bu, Color.Green, LineStyle.Solid, 2 ); PlotSeries( p, be, Color.Red, LineStyle.Solid, 2 ); PlotSeries( p, n, Color.DarkBlue, LineStyle.Solid, 1 ); PlotSeries( p1, s, Color.Blue, LineStyle.Histogram, 2 ); } } else PrintDebug( Bars.Symbol + " does not contain named series" );



Other sentiment data

Since 1958 NYSE publishes a monthly Factbook on Securities Market Credit data. It is comprised of the data for Margin debt, Free credit cash accounts, and Credit balances in margin accounts. The Quandl symbol is NYXDATA/MARKET_CREDIT. For more information you may refer to this article: NYSE Margin Debt As An Indicator Of Long-Term Movements In S&P 500.

The data is supported as Named DataSeries only (so the chart appears blank):


This example illustrates how to plot the NYSE Factbook named DataSeries. It can only be used with the Quandl symbol NYXDATA/MARKET_CREDIT:

if( Bars.Symbol == "NYXDATA/MARKET_CREDIT" )
{
	if( Bars.HasNamedDataSeries )
	{
		DataSeries md = Bars.FindNamedSeries("Margin debt");
		DataSeries fc = Bars.FindNamedSeries("Free credit cash accounts");
		DataSeries cb = Bars.FindNamedSeries("Credit balances in margin accounts");
            
		if(md != null && fc != null && cb != null)
		{
			ChartPane p = CreatePane( int.MaxValue, true, true );
			PlotSeries( p, md, Color.Red, LineStyle.Solid, 2 );
			PlotSeries( p, fc, Color.Green, LineStyle.Solid, 2 );
			PlotSeries( p, cb, Color.Violet, LineStyle.Solid, 1 );
		}
	}
	else
		PrintDebug( Bars.Symbol + " does not contain named series" );
}



Bitcoin data

If interested in Bitcoin, the symbols for BTC/USD are: BCHARTS/BITSTAMPUSD and BCHAIN/MKPRU. There are some supporting time series that the provider can recognize like number of Bitcoin users or exchange trade volume which are available in datasets like Bitcoin Charts Exchange Rate Data and Blockchain.

Issues and limitations