Data provider alternatives with dividend-adjusted data?

ruediger78

#1

5/19/2017 6:13 AM

Since Yahoo data provider doesn t work anymore, I wonder if there are any other free or paid data providers, that offer dividend adjusted data.

Eugene

#2

5/19/2017 6:35 AM

Re: Yahoo: "doesn t work anymore" = "doesn't work yet".

ruediger78

#3

5/19/2017 7:24 AM

Ok, but any other options meanwhile?

Here's a routine that you can use to adjust Wealth-Data for dividends. Remember, if you're using dividend-adjusted data, make sure to disable "Apply dividends to backtest results.." in the Preferences (F12) > Backtest Settings.

CODE:
Please log in to see this code.

Size:

Color:

Here's a routine that you can use to adjust Wealth-Data for dividends. Remember, if you're using dividend-adjusted data, make sure to disable "Apply dividends to backtest results.." in the Preferences (F12) > Backtest Settings.

[code]using System;
using System.Collections.Generic;
using System.Text;
using System.Drawing;
using WealthLab;

namespace WealthLab.Strategies
{
	public class DividendAdjustmentTemplate : WealthScript
	{
		// Note! Works only for dividends in the chart's range
		void AdjustWealthDataForDividends()
		{   
			if (Bars == null || Bars.Count == 0) 
				return;
			
			DataSeries ds = new DataSeries(Bars, Bars.Symbol + "(Dividend Adjustment)"); 
			ds += 1;
			
			IList<FundamentalItem> fList;
			try 
			{
				fList = FundamentalDataItems("Dividend (Wealth-Data)");
				if (fList.Count == 0) return;
			}
			catch { return; }

int bcm1 = ds.Count - 1;
			
			// Assign the dividend correction to the day prior to ex-date
			// Adustment factor is 1 - dividend/close
			for (int n = fList.Count - 1; n >= 0 ; n--)
			{            
				FundamentalItem fi = fList[n];
				if( fi.Bar > 0 )
				{
					ds[fi.Bar - 1] = 1 - (fi.Value / Close[fi.Bar - 1]);
				}
			}   
         
			for (int bar = ds.Count - 2; bar >= 0 ; bar--)
			{   
				ds[bar]    *= ds[bar + 1];
				PrintDebug(bar + "\t" + ds[bar].ToString("0.0000"));
				Open[bar]  *= ds[bar];
				High[bar]  *= ds[bar];
				Low[bar]   *= ds[bar];
				Close[bar] *= ds[bar];
			}
			
//			ChartPane adjPane = CreatePane(50, true, true);
//			PlotSeries(adjPane, ds, Color.Blue, LineStyle.Dots, 3);
		}
	
		
		protected override void Execute()
		{
			// Call this just once at the beginning of the script
			AdjustWealthDataForDividends();
		}
	}
}[/code]

Eugene

#5

5/19/2017 8:56 AM

Re: other options. Morningstar and Stooq may be dividend-adjusted but you'd want to check out yourself.

ruediger78

#6

5/20/2017 2:58 AM

Thanks Cone and Eugene for your ideas. To be more precise I need the data for these symbols: AGG BIV BLV BND BSV BWX CIU CSJ EDV GVI IEI ITM JNK LQD MBB MUB PCY SHM SHY STPZ TFI TIP TLT VCIT VCLT VCSH VGIT VGLT VGSH VMBS WIP
These are bond ETFs on which I do a rotation strategy. Some of these have very high dividend yield compared to their movement. So to measure the performance it is really important to include the dividends in the calculation.
For now I tried Morningstar, Quotemedia, Nasdaq. All of these have data, but its not dividend adjusted. When trying to download Wealth Data for 8 sysmbols I get the message, that those are not part of my subscription. For the rest there is no data found. Stooq doesnt have data for any symbol.
Any other ideas?

Cone

#7

5/20/2017 10:03 AM

Wealth-Data just covers the Dow 30, S&P 100, and Nasdaq 100 stocks. Sorry we can't help you there.

Until the dust settles at Yahoo!, for the next week or two, you can update your data by hand using the Bar Data Editor. It's a little painful for that many symbols, but just open a chart for each symbol, double click anywhere in the chart area, and then add a new bar(s) by entering the data - ensure that you specify the new Date.

There's no way to enter dividends this way, but maybe this will get you by for the time being.

Note!
You can actually use any EOD source that you wish for pricing and apply the dividends using the method I gave above. However, since Wealth-Data doesn't have the dividends for those symbols, make this change:

CODE:
Please log in to see this code.

innertrader

#8

6/28/2017 8:45 PM

Is there a reason there is not an option in the Data Manager to adjust Fidelity data for dividends?

Eugene

#9

6/28/2017 8:57 PM

@ ruediger78

QUOTE:
For now I tried Morningstar, Quotemedia, Nasdaq. All of these have data, but its not dividend adjusted.

Although Yahoo's data quality with regard to splits seems to have noticeably improved after last weekend's breaking change (to be fixed shortly) -- to the extent of making me think that "Ignore selected splits" option is becoming superfluous -- we're about to release a new data provider with split- and dividend-adjustment applied on-the-fly as in the Yahoo provider. Like MSN did before and Wealth-Data does now, I hope it serves as a backup for Yahoo because it will cover the whole universe of U.S. stocks.

@ innertrader

QUOTE:
Is there a reason there is not an option in the Data Manager to adjust Fidelity data for dividends?

Because it does not support dividend adjustment. Dividends are added/subtracted to/from the returns if "Apply dividends to backtest results..." is enabled in Preferences.

innertrader

#10

6/30/2017 3:35 AM

@Eugene

If I understood your previous explanation (in a different thread), you get Raw (unadjusted) data from Yahoo and the Data Manager does the adjustments for dividends and splits, user option selectable, presumably because you also get a separate download from Yahoo (or someone) that lists historical splits and dividends for each symbol. Why can't you do the same thing with the Fidelity data? Is there something different (i.e., not Raw) about the data that you get from Fidelity that makes this technically impossible or impractical?

Eugene

#11

6/30/2017 5:53 AM

QUOTE:
Why can't you do the same thing with the Fidelity data?

There must be a good reason that the Fidelity data providers work the way they work.

Our third party company MS123 LLC runs the wealth-lab.com website, is licensed by Fidelity to support and resell Wealth-Lab products to international customers. We are not involved in preparing the Fidelity Data Providers as well as are not responsible for developing the main Wealth-Lab application. Besides other activities, MS123 takes care of the website, support, Extension development, and acts as an analyst and facilitator to submit problems to Fidelity.

Cone

#12

6/30/2017 11:04 PM

Not sure if I posted this in another thread, but you can adjust Fidelity data yourself by using the 'AdjustForDividends' routine below.

The routine is set to adjust only for regular dividends (see comments in the code). Other dividends may or may not already be adjusted in the Fidelity data. In other words, there's a chance that some special dividends are already adjusted.

CODE:
Please log in to see this code.

Size:

Color:

Not sure if I posted this in another thread, but you can adjust Fidelity data yourself by using the  'AdjustForDividends' routine below.

The routine is set to adjust only for regular dividends (see comments in the code).  Other dividends may or may not already be adjusted in the Fidelity data. In other words, there's a chance that some special dividends are already adjusted.

[code]using System;
using System.Collections.Generic;
using System.Text;
using System.Drawing;
using WealthLab;

namespace WealthLab.Strategies
{
	public class DividendAdjustmentMockup : WealthScript
	{
		StrategyParameter _adj;
		
		public DividendAdjustmentMockup()
		{
			_adj = CreateParameter("Adjust=1", 1, 0, 1, 1);
		}
		
		// Note! Works only for dividends in the chart's range
		void AdjustForDividends(string DividendPerShareItem)
		{   
			if (Bars == null || Bars.Count == 0) 
				return;
			
			DataSeries ds = new DataSeries(Bars, Bars.Symbol + "(Dividend Adj)"); 
			ds += 1;
			
			IList<FundamentalItem> fList;
			try 
			{
				fList = FundamentalDataItems(DividendPerShareItem);
				if (fList.Count == 0) return;
			}
			catch { return; }
			
			// Assign the dividend correction to the day prior to ex-date
			// Adustment factor is 1 - dividend/close
			for (int n = fList.Count - 1; n >= 0 ; n--)
			{            
				FundamentalItem fi = fList[n];
				if( fi.Bar > 0 )
				{
					// Remove the test for "CD.R" (regular divs) if you want to adjust for all 
					// dividends types;  e.g., CD.X (special), CD.PS, CD.CG, CD.OP, CD.US15, CD.NU
					if (fi.GetDetail("DPC").Contains("CD.R"))
						ds[fi.Bar - 1] = 1 - (fi.Value / Close[fi.Bar - 1]);
				}
			}
         
			for (int bar = ds.Count - 2; bar >= 0 ; bar--)
			{   
				ds[bar]    *= ds[bar + 1];
				//				PrintDebug(bar + "\t" + ds[bar].ToString("0.0000"));
				Open[bar]  *= ds[bar];
				High[bar]  *= ds[bar];
				Low[bar]   *= ds[bar];
				Close[bar] *= ds[bar];
			}			
//			ChartPane adjPane = CreatePane(50, true, true);
//			PlotSeries(adjPane, ds, Color.Blue, LineStyle.Dots, 3);
		}
	
		
		protected override void Execute()
		{
			if (_adj.ValueInt == 1)
				AdjustForDividends("dividend");  // Call this to adjust
		}
	}
}[/code]

innertrader

#13

7/2/2017 12:54 AM

Thanks, Cone. Is the line:

CODE:
Please log in to see this code.

...where the dividend data is captured in the program? I presume this means that the historical dividend data is already available as part of the data downloaded for each symbol? If so, wouldn't the use of this class and using Fidelity for historical data obviate the need for Yahoo data? It seems that Fidelity data is much more reliable / available without the format surprises and unannounced changes of Yahoo. Has that been your experience?

I'm a bit unclear how to apply the information and code you provided in #7 above (following "Note!"), in light of my question. Is it getting dividend data from Wealth-Data?

Cone

#14

7/3/2017 4:04 AM

Fidelity dividend (and split) data are available just by downloading Fidelity price data. As in the example, "dividend" is the string to pass to Fundamental data functions to access Fidelity dividend per share data. If you want Yahoo! Dividend data, the string is "Dividend (Yahoo! Finance)". For Wealth-Data it's "Dividend (Wealth-Data)".

Fidelity data are reliable (after 2001 or so), but don't update it for the last session before 6am the next day to ensure that you get the after-market corrections. For more on that see this post.

Also, before doing serious backtesting with Fidelity data, run the Bad History Data Check script (or similar) on the test range. Fidelity data tends to delete chunks of data "spontaneously" on occasion, in which case needs to be refreshed.

Eugene

#15

7/3/2017 7:20 AM

QUOTE:
run the Bad History Data Check script (or similar)

For innertrader, here's the similar GUI tool:

Install Data Tool
Online user guide

ruediger78

#16

11/17/2017 9:19 AM

Just realized, that dividend adjusted data is messed up in Yahoo for several symbols. Added a screenshot. I downloaded data at the 3 arrows. Data is good until the closest dividend payout date. Then the distortion beginns. So it seems that a miscalculation happens with adjusted data. Is it a problem on your side? Or a problem with the Yahoo data, that cant be corrected by you?

BB.png

Eugene

#17

11/17/2017 9:27 AM

There are positively no issues with or w/o dividend adjustment applied to Y! data. See below:

Making sure that Y! provider is up-to-date (which I think already is - or the company name would be missing from your screenshot), reopen the chart window. Don't change the dividend/split adjustment setting on-the-fly yet. Restart WLD. If nothing helps you might want to create a support ticket and we'd look into it more closely.

biv.png

Eugene

#18

11/19/2017 9:19 PM

Just in case someone else stumbles onto the issue, "Reload chart history" fixed the problem for ruediger78.

bliss

#19

11/25/2017 5:57 PM

I'm an inexperienced user and new to the forum.
Having trouble getting dividend adjusted data into data history for selected DOW components.
Generated a partial data set using Y! as data source, but system returned error updating message...
"The remote server returned an error (403) forbidden"
For each symbol got "the remote name could not be resolved: ichart.finance.yahoo.com"
Any suggestions?

Eugene

#20

11/25/2017 6:04 PM

Hi William,

Welcome to the forum. I think you may be using an outdated version of the Y! provider. The free data feed experienced a number of breaking changes this year so it's important to have the latest version installed. To do it please open up Tools > Extension Manager, update Yahoo! provider to its latest version and restart WLP.

Eugene

#21

2/29/2020 7:00 AM

QUOTE:
For now I tried Morningstar, Quotemedia, Nasdaq. All of these have data, but its not dividend adjusted

Starting from v2020.03, the data by QuoteMedia provider is split- and dividend-adjusted. You can disable the adjustment on synonymous tab in the Data Manager.

Note to all posters:

This thread has been "hijacked" a number of times and thus deviated from its true subject which is "Data provider alternatives..." Any new posts like #16 or #19 which concern troubleshooting Y! or any other provider will be removed from this thread without notice. Thanks for picking a more closely related thread for your questions.

NorgateData

#22

3/30/2020 7:13 AM

The Norgate Data extension also allows you configure your view of the data with or without various types of adjustments (including dividends):

Here's a screenshot of how this done via the Data Manager within Wealth-Lab: