Data Manager stopped updating Fidelity EOD data (only)

cmtn

#1

12/2/2017 4:15 PM

My daily symbols are not updating when I click the "Update all data for selected Providers now" in the Data Manager. This stopped working a few days ago and I don't know why.

The minute and 10 minute symbols are updating. Nothing is in the log about the daily symbols.

I can update individual sets one at a time, but this is tedious.

I use the Fidelity data provider.

I am logged in.

I am using version 6.9.17.0.

Thanks

Eugene

#2

12/2/2017 4:33 PM

Hi,
In the first place, please consider updating to v6.9.19.

Having encountered an error message or an issue, please always check out these page before reporting it:

1. Known Errors
2. Open Issues

In particular, this issue is known and could be resolved by following one of workarounds here:

Open Issues > Fidelity data can not be updated except for...

cmtn

#3

12/7/2017 2:47 AM

How do I get v6.9.19?

If I click Help...Software Upgrade, I get this message: "The installed version of Wealth Lab Pro is current."

My About screen says I'm on 6.9.17.

Thanks

Eugene

#4

12/7/2017 6:41 AM

Craig,

6.9.19 aside, have you been able to fix the Fidelity data update?

cmtn

#5

12/17/2017 1:51 AM

I deleted all of my data and it worked for a week. And then it stopped.

How do I get v6.9.19?

Eugene

#6

12/17/2017 7:53 AM

QUOTE:
How do I get v6.9.19?

I'm sure there was a forum thread where Cone laid out the upgrade procedure. I lost track of it as I don't work with WLP. Not sure the upgrade would help, though.

QUOTE:
I deleted all of my data and it worked for a week. And then it stopped.

Another reverse split, most likely. We can use this calendar to detect them:

Stock Split Calendar at Fidelity

To help find those symbols and reload their data, here's a Strategy to help find reverse splits for a given month in your current DataSet. Point it at any symbol (but don't run in portfolio mode!) and it will spit out all symbols from your DataSet which had reverse splits this month. Drag the parameter slider to change months.

Prerequisites:
1. Community Components installed to get required HtmlAgilityPack dependence (restart WLP after installation if not installed already)
2. In addition to what's already checked there, check System.Xml, System.Web and System.Core on .NET Framework tab of the strategy's "References..." dialog

Compile it, and if you don't see a "Strategy FidelityReverseSplitCalendar compiled successfully!", you did something of that wrong.

CODE:
Please log in to see this code.

Size:

Color:

[quote]How do I get v6.9.19? [/quote]
I'm sure there was a forum thread where Cone laid out the upgrade procedure. I lost track of it as I don't work with WLP. Not sure the upgrade would help, though.

[quote]I deleted all of my data and it worked for a week. And then it stopped.[/quote]
Another reverse split, most likely. We can use this calendar to detect them:

[link=https://eresearch.fidelity.com/eresearch/conferenceCalls.jhtml?tab=splits]Stock Split Calendar[/link] at Fidelity

To help find those symbols and reload their data, here's a Strategy to help find reverse splits for a given month in your current DataSet. Point it at any symbol (but don't run in portfolio mode!) and it will spit out all symbols from your DataSet which had reverse splits this month. Drag the parameter slider to change months.

[b]Prerequisites:[/b]
1. [link=https://www.wealth-lab.com/Extensions/Details/10]Community Components[/link] installed to get required [b]HtmlAgilityPack[/b] dependence (restart WLP after installation if not installed already)
2. In addition to what's already checked there, check [b]System.Xml, System.Web[/b] and [b]System.Core[/b] on [i].NET Framework [/i]tab of the strategy's "References..." dialog

Compile it, and if you don't see a [color=#43a800]"Strategy FidelityReverseSplitCalendar compiled successfully!"[/color], you did something of that wrong.

[code]using System;
using System.Collections.Generic;
using System.Text;
using System.Drawing;
using WealthLab;
using System.Linq;
using System.Net;
using HtmlAgilityPack;
using System.Web;

namespace WealthLab.Strategies
{
	public class ReverseSplit
	{
		public string Symbol { get; set; }
		public string SplitRatio { get; set; }
		public string ExDate { get; set; }
	}
	
	public class FidelityReverseSplitCalendar : WealthScript
	{
		private StrategyParameter paramMonth;
      
		public FidelityReverseSplitCalendar()
		{
			paramMonth = CreateParameter("Month",12,1,12,1);
		}

public List<ReverseSplit> GetListOfReverseSplitForDate()
		{
			var lstReverseSplits = new List<ReverseSplit>();
			var currMonth = paramMonth.ValueInt;
			var currYear = DateTime.Today.Date.Year;
			
			string URI = string.Format(@"https://eresearch.fidelity.com/eresearch/conferenceCalls.jhtml?tab=splits&begindate={0}/1/{1}",
				currMonth, currYear);
			string xpath = "//table[@id='messageTable4']//tr[descendant::td]";

var lstReverseSplit = new List<ReverseSplit>();
			HtmlWeb web = new HtmlWeb();
			HtmlDocument doc = web.Load(URI);

if (doc.DocumentNode != null)
			{
				try
				{
					var row = doc.DocumentNode.SelectNodes(xpath);
					for(int i = 0; i < row.Count; i++)
					{
						var item = row[i];
						
						var symbol = item.Descendants("td").ToArray()[0].InnerText.Trim().ToString();
						var splitRatio = item.Descendants("td").ToArray()[1].InnerText.Trim().ToString();
						var exDate = HttpUtility.HtmlDecode(item.Descendants("td").ToArray()[4].InnerText.Trim().ToString());
						
						var _s = splitRatio.Split(new char[]{':'});
						if( _s.Count() > 0)
						{
							if( double.Parse(_s[0]) < double.Parse(_s[1]) )
							{
								lstReverseSplits.Add( new ReverseSplit(){Symbol = symbol, SplitRatio = splitRatio, ExDate = exDate } );
							}
						}
					}
				}
				catch (Exception)
				{
					throw;
				}
			}

doc = null;
			return lstReverseSplits;
		}
      
		protected override void Execute()
		{
			var splitsThisMonth = GetListOfReverseSplitForDate();
			
			PrintDebug( "Showing only reverse splits:\r\n" );
			PrintDebug( "Symbol" + "\t" + "Split Ratio" + "\t" + "Ex-Date" + "\r\n" );
			
			for(int ds = 0; ds < DataSetSymbols.Count; ds++)
			{
				var sym = DataSetSymbols[ds];
				
				foreach (var s in splitsThisMonth)
				{
					if(sym == s.Symbol)
						PrintDebug (s.Symbol + "\t" + s.SplitRatio + "\t" + s.ExDate);
				}
			}
		}
	}
}[/code]

superticker

#7

12/17/2017 4:27 PM

QUOTE:
I deleted all of my data and it worked for a week. And then it stopped.

QUOTE:
Another reverse split,...

You may want to look at http://www2.wealth-lab.com/WL5Wiki/OpenIssues.ashx under the "Fidelity data" topic for more background about that.

I run into these stock-split issues with ETFs all the time. I wish Fidelity would just fix this Data Manager bug--but that would be too easy. :-) What I do is look through the last Data Manager log for a "split" followed by a termination of the update for that DataSet. That will identify what stock (or ETF) split. Then I delete that ticker symbol from that DataSet and run the update again.

The problem with this approach is that you may have to repeat doing the above several times for each symbol that recently took on a split. So using the code in the previous post might identify all the symbols that had splits altogether so you can delete them all at once, then rerun the Data Manager update.

Once Data Manager has sanitized all the split symbols you deleted, you can re-add those symbols (I typically don't bother) so Data Manager reloads them from scratch on the next update.

cmtn

#8

12/24/2017 3:29 PM

Christmas break finally gives me time to work on this.

It seems to be working now by cleaning up the splits per above from Eugene and superticker.

Thanks everyone.