Is there any possibility that Intrinio could be added as a provider of fundamental data?

I think their solution compares somewhat favourably with Zacks Fundamental B, and it is about 2/3 of the price for the 10-year version that allows 25,000 requests per day.

Thanks for your enhancement request.

I was trying to find some selling points for a possible Intrinio data provider suite. They provide real-time intraday data but also have some drawbacks. Like it does have security fundamentals but none so for foreign markets, or their historical company news are sourced from Yahoo and I had some concerns before etc. But overall Intrinio looks like an interesting competitor to Quandl.

If you find their fundamental data worthwhile this might be something for me to start with. Let's focus on the fundamentals for now. I cannot make promises or set an ETA right away but I've put your provider request on my product backlog.

P.S. What I dislike is that new accounts require phone verification - even for their rate-limited free trials.

Yes, the phone verification seems a little over the top.
I have been using the free trial into Excel, and the quality seems quite good.

Thanks for adding it to the backlog :)

Peter, I've started looking into the Intrinio fundamentals. While the idea seemed interesting at first there are other solutions we have to consider before diving into it. Especially considering that I don't find their API very convenient.

Firstly, have you explored the YCharts paid data plans already? How does it compare to Intrinio or Zacks? The good thing about YCharts is that we could add fundamental items to our data provider upon request (if offered by YCharts). For comparison, below are the Intrinio item names I've extracted from their API:

operatingrevenue
totalrevenue
operatingcostofrevenue
totalcostofrevenue
totalgrossprofit
sgaexpense
rdexpense
totaloperatingexpenses
totaloperatingincome
otherincome
totalotherincome
totalpretaxincome
incometaxexpense
netincomecontinuing
netincome
netincometocommon
weightedavebasicsharesos
basiceps
weightedavedilutedsharesos
dilutedeps
weightedavebasicdilutedsharesos
basicdilutedeps
cashdividendspershare

cashandequivalents
shortterminvestments
notereceivable
accountsreceivable
netinventory
othercurrentassets
totalcurrentassets
netppe
longterminvestments
othernoncurrentassets
totalnoncurrentassets
totalassets
shorttermdebt
accountspayable
accruedexpenses
totalcurrentliabilities
longtermdebt
othernoncurrentliabilities
totalnoncurrentliabilities
totalliabilities
commitmentsandcontingencies
commonequity
retainedearnings
aoci
totalcommonequity
totalequity
totalequityandnoncontrollinginterests
totalliabilitiesandequity
currentdeferredrevenue
noncurrentdeferredrevenue

netincome
netincomecontinuing
depreciationexpense
noncashadjustmentstonetincome
increasedecreaseinoperatingcapital
netcashfromcontinuingoperatingactivities
netcashfromoperatingactivities
purchaseofplantpropertyandequipment
acquisitions
purchaseofinvestments
saleofinvestments
otherinvestingactivitiesnet
netcashfromcontinuinginvestingactivities
netcashfrominvestingactivities
repaymentofdebt
repurchaseofcommonequity
paymentofdividends
issuanceofdebt
issuanceofcommonequity
otherfinancingactivitiesnet
netcashfromcontinuingfinancingactivities
netcashfromfinancingactivities
netchangeincash
cashinterestpaid
cashincometaxespaid

Also, the Intrinio API returns "Reported" and "Standardized" fundamentals. Do you know if there's any substantial difference?

Hi Eugene - thanks for looking at this. A couple of points:

Reported/Standardized

To me the distinction between "As Reported" and "Standardized" is quite important.

The "As Reported" data is the exact number that the issuer reports in the 10Q/K. The "Standardized" number is changed to make it comparable to other companies. XBRL is a great idea - but there are many issues when comparing companies. For example, something as simple as revenue is not necessarily reported the same way across companies. Bloomberg also works with both As Reported and Standardized data.

Intrinio's own explanation is here: http://blog.intrinio.com/normalized-xbrl-data/

In my case, I would certainly prefer to work with standardized data.

Data Items

I think Intrinio has many more data items than the ones you listed. If you look at https://intrinio.com/financial-data/us-fundamentals-financials-metrics-ratios?method=api - and click on the Data Explorer tab, it lists in the menu on the left what data items there are. I see 356 items there.
Clearly, some of them are calculated, which we could do ourselves, but it requires only one call to get a calculated ratio, but several to do it yourself. Probably less error prone as well. In my particular case, Intrinio certainly has all the data items that I need, YCharts are missing some, although I could probably find most of the data needed to calculate them.

To me, it is also important that Quandl and Intrinio can deliver stock price data, which I can use in Excel.

Pricing

When comparing YCharts, Quandl and Intrinio I cannot quite find what the download/api-call limitation is for Quandl, but the $40/month version of Intrinio allows 5.000 API calls per day, with no limitation on download size. 25.000 calls for the $100 version. The Intrinio package also includes historical price data that I can feed into Excel.

From your explanation in the YCharts thread, it sounds like you have to download all of the YCharts data for each stock every time you need access so a lot of unnecessary data comes with it. If it is possible to make individual calls to the Intrinio API would that not mean that the 5000 calls/day gives much more room than 500mb/month?

Quandl has most of the data items I need, but seems expensive/risky at $1.800 per year. Paying monthly clearly makes it less risky if for some reason it does not fulfill requirements.

Peter, thanks for your detailed feedback.

QUOTE:
To me, it is also important that Quandl and Intrinio can deliver stock price data, which I can use in Excel.

Unless they offer some unique data, you have to convince me that the creation of Intrinio static data provider would pay off. What stock data that you have at Intrinio cannot be found with Quandl or elsewhere?

QUOTE:
I see 356 items there. Clearly, some of them are calculated,

I stand corrected, thanks. Clearly I was doing something wrong yesterday when I was exploring the API. There are 121 items in the "calculations" group. Also as you point out, my list from yesterday is crippled because I overlooked Intrinio's forced pagination (yes, even for the item names) which cannot be turned off.

Each fundamental data item in this API has to be requested individually i.e. total number of items per each symbol updated. More so if an item has over 100 fundamental data points (or 1000 for historical data) one has to care about pagination. On top of it, another drawback is the lack of bulk requests - you don't seem to be allowed to request data for multiple companies (or different data items) in one shot.

QUOTE:
From your explanation in the YCharts thread, it sounds like you have to download all of the YCharts data for each stock every time you need access so a lot of unnecessary data comes with it.

In one of its previous releases I have optimized the YCharts quota usage considerably. Anyway, for all fundamental data providers Wealth-Lab will request all the fundamental items. Fine-grained selection may currently be possible only with custom data providers. One way to minimize the YCharts quota consumption for 'lite' plans is to update the fundamental data on a weekly/monthly basis.

QUOTE:
YCharts are missing some, although I could probably find most of the data needed to calculate them.

Could you please clarify which ones? I'm asking because it might be more practical and take less time to add support for missing fundamental items from YCharts' vast supply on demand than start the new provider development. For the moment I'd like to consider Intrinio as a 'last resort' until the other options are exausted.

QUOTE:
you don't seem to be allowed to request data for multiple companies (or different data items) in one shot.

I think this is explained in this article http://help.intrinio.com/api-only-help/what-is-an-api-call-and-how-is-it-counted - in the section named "How do you count API calls" it says: A developer can get the entire income or balance sheet for a company in a single API call.

However, if making what they call dimensional calls - i.e. several companies or data items in the same call, that would count as more than one call. Still - if we can get the entire balance sheet in one single call, that would give us many data items as long as we can figure out how to parse it.

QUOTE:
Anyway, for all fundamental data providers Wealth-Lab will request all the fundamental items.

I think I need to understand better how Wealth-Lab accesses fundamental data from these providers - and I need to think about what it is I am trying to do.

Ideally I want to build a strategy that will run on a dataset of for example 100 stocks using single-symbol mode, going back let us say 12 years in time using daily bars. This will give me roughly 3200 bars, and 320,000 runs of the datasetsymbol loop. In each of those runs, I need access to 15-20 individual data items. If I have direct access to calculations like earnings growth and ROE growth and their variation coefficients, I would probably only need it for the most recent time period, otherwise I would need to go back in time as well to calculate growth numbers myself.

If one data item takes up 10 bytes, this would mean that if individual calls are made every time the loop runs, it would require at least 6.4mm calls and 64mb of data. Clearly that would be impossible with Intrinio and would quickly use up a 500mb monthly limit.

However, companies only report once a quarter, so if I can find a way to check when new earnings are reported and then only download them then, and also find a way to store the data once I have downloaded it for access in the later bars, and I also assume that I can call an entire balance sheet or income statement in one call, then it could probably be cut down to maybe 24,000 calls and only 96K of data.

Still - if I am thinking about this the right way - it seems like a pricing plan based on individual API calls is just not feasible, and I would have to find a provider that is based on data size limits.

But is this how it works? - If I have 100 stocks in my dataset and this is connected to for example YCharts what does Wealth-Lab actually download? - Do you download it once and then my strategy can access it all it wants? - and how does that work with respect to time periods? - Is what I am describing above even at all possible with any of these providers?

QUOTE:
Could you please clarify which ones?

I cannot see what data items are available from YCharts (other than the ones you posted) but I think Quandl has all the data that I need, including some very tasty growth numbers in their Expanded Fundamental Ratios. With them, it is mainly a question of price because they only have a yearly plan. $1,800 up front for just 10 years of data is a lot for something that I do not even know for sure if it will work. And I cannot find their data download limit anywhere.

By the way - have you looked into just getting some of this data directly from xbrl.us ? That is free and certainly possible from within Excel - the main problem is standardization as mentioned above.

QUOTE:
I think this is explained in this article http://help.intrinio.com/api-only-help/what-is-an-api-call-and-how-is-it-counted - in the section named "How do you count API calls" it says: A developer can get the entire income or balance sheet for a company in a single API call.

Thanks for the pointer. This way doesn't seem very much useful to me or there's a bug as it doesn't support Quarterly fundamentals. If you try the example link with the following adjustments and vary the fiscal_period from Q1 to Q4 it would return exactly same figures. If I were their customer I'd consider raising an issue with them:

https://api.intrinio.com/financials/standardized?identifier=AAPL&statement=income_statement&fiscal_year=2017&type=QTR&fiscal_period=Q4

The same document advertises being able to make "dimensional calls" (i.e. API call with multiple tickers / data points). While this works for price it does not for the fundamental data. So it's useless in the context of my task.

I see how to employ the Intrinio API, just pointing at its drawbacks. No big deal.

QUOTE:
If one data item takes up 10 bytes, this would mean that if individual calls are made every time the loop runs, it would require at least 6.4mm calls and 64mb of data. Clearly that would be impossible with Intrinio and would quickly use up a 500mb monthly limit.

Nope. When you run a fundamental data update you make ~300 data items * 100 stocks * year (let's say Intrinio returns 10 years of data) = 300000 API calls. Actually, there will be more requests due to pagination but that's not a very significant burden. This data are cached by Wealth-Lab on your disk until next manual/scheduled fundamental data update. Unless you have On Demand Update enabled (which makes unscheduled requests to the API), and I'd recommend that you don't, you'd only consume 300K API calls per update.

Note that the hypothetical provider would request all the key stats (let's say there are 300 but I'd reduce the number to a more manageable figure - say 100-150), not your 20 items of choice. I briefly touched this in post #6. Still nowhere near "6.4mm calls".

Being able to request all the key items in one call would greatly improve provider's speed and reduce the number of API hits but this seems broken for Quarterly data as I supposed above.

QUOTE:
By the way - have you looked into just getting some of this data directly from xbrl.us ?

Disclaimer: haven't seen how the Excel addon works. It seems to me that XBRL requires too much extra processing for us mere mortals.

QUOTE:
I cannot see what data items are available from YCharts

You can browse some of them here:
https://ycharts.com/companies/AAPL/key_stats

Before website redesign there was an item browser/catalog that I cannot locate now which showed hundreds of fundamental items/calculations and countless economic items. This Excel sheet contains 4,029 fundamental items from which I could add to the YCharts provider on demand (quota wise):

https://ycharts.com/securities/export_metric_reference_guide/excel

You might want to start a 7-day free trial. There's an Excel addin (and now even a .NET API since last year, hooray!) The data goes back up to 30 years.

So - if I understand you correctly - if I subscribe to the YCharts Pro version, I would then download (100 stocks * 48 quarters * 125 (all) data items * 10 bytes) = 6mb of data, once, and then I can access that data from my strategy as much and as many times as I want, until I decide to update it again.

That sounds sensible, if that is correct. I would be able to update the data once a day, and still stay within a 500mb/month limit. I could even increase the number of stocks, which I might want to do.

I agree with you that Intrinio (or indeed any provider with API-call based pricing) does not make sense for this. I also hear you on the additional problems with Intrinio. And on the XBRL problems - standardization becomes a huge problem. That is pretty much the main reason why any of these vendors have a business.

I will start the 7-day YC trial next wednesday (when I have a few days to actually try it out).

And you would be willing to add some data items that I need to the list that is downloaded from YCharts?

QUOTE:
So - if I understand you correctly - if I subscribe to the YCharts Pro version, I would then download (100 stocks * 48 quarters * 125 (all) data items * 10 bytes) = 6mb of data, once, and then I can access that data from my strategy as much and as many times as I want, until I decide to update it again.

Your understanding is correct, except that the actual download size may vary (and most likely both the consumption of "quarters" and "bytes" params will be higher).

QUOTE:
And you would be willing to add some data items that I need to the list that is downloaded from YCharts?

For sure. You can request them in its dedicated forum thread, for example. Please note that I'll keep from including items which lead to big consumption of the download quota like EOD data and/or series in the bullet list under "Available fundamental items" here: YCharts provider's online user guide.

Please take your time with the trial and let me know if YCharts would suit you. If not we'll revisit Intrinio.