This is a Wealth-Lab static and streaming provider for free web-based daily data. It's possible to download the data for U.S. and world stocks, indexes, funds and more.

BBFree static/streaming Daily data provider - installation

For more information, please visit project's home page:

BBFree static/streaming Daily data provider in the Wealth-Lab Wiki

Is there any way to make updating the daily data faster?

The benefit is that you could find really rare/international symbols not available with Yahoo, MSN or Google providers.

Unfortunately, with this vendor a pretty sizable chunk of data must be downloaded and processed just to update the daily data for a single symbol (unlike Yahoo!)

I'm not sure if even adding multi-threaded downloads would help this, but will mark it for checking later.

Yahoo is a good provider but it does not have all the penny stocks.

Google does have penny stocks but with a lot of errors and missing data in between days.

MSN has all the penny stocks, but it gives errors and skips many stocks while updating the database. Therefore, it is not a reliable source. It also takes a lot of time to update it.

BBFree looks like have all the stocks including the penny stocks. The data seems accurate too. The only thing we need is to increase the speed of updating.

There is another source available for daily data download. http://www.quotemedia.com/
It allows multithreading, has all the stocks including the penny stocks, and the data are accurate as well. The Yahoo and Google downloader program (the blue one) has plug in for it. It would be great if Wealth Lab could make a multithreaded static adapter for this data source too.

QUOTE:
The only thing we need is to increase the speed of updating.

I've optimized the size of daily update requests so that they will consume less traffic and run faster. This is going to improve daily update speed. Will upload the new build shortly.

As to the process of creation of a DataSet/datasource and loading the data, multi-threaded downloading could help increase speed, but it's not implemented in the provider. For now, this step can be scheduled to run at night.

(If website would support gzip compression, it would speed things a bit w/o having to deal with multithreading - unfortunately, their web server doesn't support compression for the quotes, just for the rest of the site).

QUOTE:
Will upload the new build shortly.

Done.

Thanks for this BBFree provider Eugene, it's going to make it easier to keep track of that Libor spread now :)

If anyone is interested (you should be), create a BBFree DataSet with USSOC:IND and US0003M:IND, and add this script to your startup Workspace to check how the credit spreads are doing each day.

CODE:
Please log in to see this code.

Adding a strategy to the strategy monitor that has a BBFree Provider DataSet associated with it doesnt seem to update. The Symbols just remain in a "pending update state". Any ideas?

Hi Mark,

QUOTE:
Adding a strategy to the strategy monitor that has a BBFree Provider DataSet associated with it doesnt seem to update. The Symbols just remain in a "pending update state". Any ideas?

I found that the provider isn't currently supporting the Strategy Monitor.

Version 2010.06.3:

Added: Strategy Monitor support

Added three BBFree DataSets with world indices (several hundreds) to the Wiki. Thanks Robert!

While its a great addition to the providers list, there is one (big) deficiency that I hope will be addressed in the near future:

It doesn't retrieve the symbol's name/description.

So if you have d/l and are looking at a chart of, say, AMX:IND, you may be inclined to think its the US AMEX index, or maybe the stock America Movil (if your mind misses the IND suffix), etc. [AMX:IND is actually the Amsterdam Midkap Index.] Considering the number of possibilities with almost every symbol, I think its important that the symbol's full name/description also should be retrieved and made part of the file. Just like any other symbol - even other 3rd party symbols e.g. Yahoo have names attached.

The near future is perhaps a bit too optimistic, considering that the company name is absent from this data.

Then how about in the not-too-distant future? ;)

Update: version 2010.06.4 has company names for new symbols.

Note: existing data w/o company names should be reloaded for them to appear - this is because we wouldn't want to reduce speed of daily updates by running a separate HTTP request on each update!

Firstly, it was an interesting problem - otherwise I wouldn't bother wasting my weekend hours on this request. As I said, the data doesn't contain company/index names and for several reasons, we just can't take them elsewhere (like the Google provider has been quietly asking Yahoo's server): website's own symbology, a huge universe of symbols (many don't exist elsewhere), and availability of similar symbols where the suffix is what really makes the difference.

P.S. I took the courage to strip "/The" from the company names e.g. "Coca-Cola Co/The", should someone miss them, let the forum know.

Good job, Eugene!

Eugene, what's required to get the names? Refresh the data by deleting it? Recreate the DataSets?

For a handful of symbols, "Reload Chart History" works fine.

For many symbols or DataSets, you need to refresh the data by deleting it (e.g. by wiping out the ..\Data\B.......gProvider folder).

That's because we only want to download security names for new symbols, not the already existing ones. I didn't want to upset our customer wl64bit by requesting symbol names on each update - when working with a large universe of symbols like he does, running a separate request for company names could reduce daily update speed a bit.

Occasionally, names change. Perhaps the adapter can occasionally (? once a month) refresh the names automatically or pop up with a window asking if the user wants to refresh the names; OR there can be a user-specified interval at which time it will refresh the names too alongwith the price update? That way the names stay current, and it also obviates the need for deleting datasets.

No, that's too error prone and complicated. When a company name changes occasionally, just reload the chart.

QUOTE:
When a company name changes occasionally, just reload the chart.

Sure you can do that IF you know the name has changed. But since a name change would be known only rarely by the user, its best if the adapter refresh the names. Frankly, I don't see what's the big deal with it refreshing the names once in 1, 3 or 6 months.

You just can't please everyone Eugene.

However, if you're a Bloomberg customer, just purchase the Bloomberg Adapter Suite for unlimited Bloomberg Access, and, symbol names are refreshed for each update in bulk fashion. We're planning a full integration of Bloomberg (Static, Streaming, and Fundamental Providers) for Version 5 for next year.

There's no way to reliably tell when those 1, 3 or 6 months have passed, so let's forget this idea.

There's no big deal to update them in bulk fashion just like the Yahoo provider does on each update. The overhead for the Y! provider is merely a couple dozen bytes per symbol - unlike BBFree, that's why I'm concerned about how would it affect everyday update speed.

I found a workaround that will hopefully please everyone. Here's how it will work starting from the upcoming version.

So far the provider didn't need settings, and it's not practical to create a Data Manager tab just for just this option. So, the provider would create a settings file called B........FreeSettings.xml in Wealth-Lab's user data directory. It will be pretty easy to find it, having read the User Guide (Data > Where data are stored) and having enabled hidden file/folder visibility in Windows (Folder Options).

By default, updating existing symbol names will be disabled for speed reasons. To enable it, you update the BBFree provider in Extension Manager and restart Wealth-Lab, then bring up Notepad with the XML file. The one and only line that matters there will be:

CODE:
Please log in to see this code.

Change false to true, save the file, restart Wealth-Lab for the provider to read the updated settings file, and each time you go updating your DataSets it will update every existing and new symbol name.

Update: version 2010.07 with optional update of existing symbol names is out.

Thanks for this update, Eugene.

I think your fear of the time penalty may have been overblown. With the option enabled to update symbol names, it takes me just a little extra time than when the feature was not available (I d/l a few hundred symbols). And that is perfectly acceptable.

Thanks, good to know. I also need to run a test on a few thousand symbols and compare the results. Marking it for later.

Sneak peek at what's coming up.

By request of our user wl64bit, the upcoming build of the BBFree provider will have multi-threaded downloads for all symbols.

Testing on a 100-symbol dataset shows that downloading/updating has become several times faster.

As promised to wl64bit, here goes build 2010.09 with faster data download speeds.

Summary of changes:

Added: Multi-threaded data update (now 2-3x faster)
Changed: Streaming provider rewritten to comply with internal changes in WL6
Changed: possible to install in Wealth-Lab 6
Changed: internal improvements to company name extraction
Fixed: Bar data editing - some edits weren't properly saved
Fixed: GetMarketInfo

Sammy_G:

QUOTE:
I think your fear of the time penalty may have been overblown. With the option enabled to update symbol names, it takes me just a little extra time than when the feature was not available (I d/l a few hundred symbols).

Updating symbol names didn't penalize the old single-threaded provider. Unfortunately, there is a considerable time penalty when doing so with the new, mult-threaded data download routine. With regard to updating symbol names, everything stays as is.

Version 2010.11 is out - with fix for potential crash due to network issues.

BBFree provider updated to version 2011.01 on 01/12/2011.

This is mostly a maintenance release fixing some bugs. Also, the new (as of WL 6.1) GetSessionOpen method is supported - the provider now returns trading session's open price. Check out the QuickRef for details.

Update to Wealth-Lab 6.1 to be able to install and use the provider.

This is a Great tool.

As a B-g user I found that, hidden on the website ('cause you need to know the tickers, they are not "searcheable" on their website, you came with no results if you do it), are available almost all indexes, macro data, CDS etc that are also available to the professional B-g platform. Everything absolutely at no cost ! Damn, fantastic !

Thanks Eugene !!!

B-g Symbols

Why did you post this? Have you verified that this is working for the BBFree provider?

There's differerence between B. terminal and their website, and this provider (BBFree) can request only those symbols which come up when searching on their site. And that link has been available here for a long time:

What it is

Eugene this worked, for Index need add :IND

Please test with INDEXCF:IND or BDSX:IND have small error

Warning

The format of the web page for company name information has changed. When downloading the data for new symbols, you will have an error like TCust illustrated. To be fixed by upcoming build. For now, keep from adding new symbols in BBFree.

Version 2011.07 has been released with a bug fix for potential crash and screen artefacts due to company name page format change.

Please update to Developer 6.2 before installing/updating this extension.

Wl6.1 and even Wl6.2. after update to 2011.07 still crashes requesting BBFree data (in the case of SP500), at least on WL6.2 64bit. Not for DAX and ESTX but there was all data up-to-date there, so maybe this prevents crashes?

It is not helpful to bind provider updates for solving errors (especially those which crashes WL) to new releases only, at least not when the release is one day old and cannot be used by some users (like me in this case, as I coded workarounds for WL bugs in 6.1 which I was told might now be solved in 6.2. But if this is true, it requires recoding all the scripts)!

So I cannot test whether 2011.07 will work at least in WL6.2 32bit as on my production machine/license as I only installed 64bit 6.2 right now trying to get it up and running as WL crashed over night instead of providing screener data for all our screener subscribers.

See also: /Forum/Posts/WLP-Crashing-While-Running-Optimizations-31807

Current version updates the data and returns the SM updates fine.

If your experience is different with version 2011.07, then your existing data might have been corrupt. Try completely deleting all accumulated BBFree data and reloading it from scratch by deleting this folder (typical location on 7/Vista/2008 machines):

"c:\Users\hlh\AppData\Roaming\Fidelity Investments\WealthLabDev\1.0.0.0\Data\B........Provider"

I did delete BBFree data as recommended

QUOTE:
The BBFree provider must be updated to version 2011.07. Previous versions are not supported.

Current version updates the data and returns SM updates fine. If your experience is different with version 2011.07, then please completely delete all accumulated BBFree data and reload it from scratch to fix possible data file corruption

But now it crashes immediately when starting to request BBFree data (WL6.2 and 2011.07) even at the very first dataset (like the DAX as mentionde above which did not cause a crash when data was there and all of it was up-to-date).

For a timely resolution, let's move the discussion in a new support ticket where in the first post I'd like to see the exact list of DAX symbols according to their symbology. Thank you.

Hotfix:

Version 2011.07.1 has been released. It finally fixes crashes due to unexpected/missing data in security names.

On their website, there is a global directory where you can find list of companies by markets or subsectors (see this link).
As this provider has a specific codification for symbol code, is it possible to add a new feature in your extension to doxnload symbol list code with the same search criteria ?

Same for index at the following URL

And Funds are here

I think you're asking for a Classification Groups feature analogue (Yahoo provider). It's a considerable effort, I'm not prepared to answer right away.

I try to use Streaming data from this provider in the Quote tool and I don't no why but for any symbol, I have all the time same time (12:00:00 AM) and no data from Change%.

I try with AC:FP, BNP:FP, ALU:FP and AF:FP.

Currently, BBFree provider timestamps all quotes as 12:00 AM and Change $/% is not supported. Thanks for the heads-up: both to be resolved in the upcoming release.

abegy, thanks for your idea -- Classification Groups (like in Yahoo!) would be a real good improvement for a data provider with a huge symbol universe like BB. Having just finished populating my SQL database with tens and hundreds of thousands of funds, indexes, and stocks from BB website, I saw the following:

158295 funds
16593 indexes
92558 stocks (by sector)
93700 stocks (by country)

After finding the best scheme to optimize the size of downloadable classification file I'll try adding Classification Groups to the BBFree provider in an upcoming release.

Thank you very much to have taken into account this idea.

As you have seen, the BBFree provider has a specific codification : there is a suffix to be able to specify from which country is the symbol or the type of the symbol (IND for index).

It could be a good thing that your extension manages symbol translating table. The idea is to give us the possibility to match symbol code with one of your fundamental data provider extension :)

The BBFree provider, like any other, works with its own symbology. Providers do not need no "symbol translation tables" because each one is independent and "doesn't care" about other provider's symbology. This is the design with which we have to live with.

Ideally, it's the application that should be designed to support a matching table but in this case, I would not nourish a hope that such significant afterthought with such less "bang for the buck" would be ever added to Version 6.

BBFree comes out with its most noticeable update in months: meet version 2011.11.

Highlights:

* Added: Classification Groups for stocks, indexes, and funds
* Added: Market Manager support
* Added: support for Change in Quotes window (found by abegy)
* Fixed: Time in Quotes window (found by abegy)
* Fixed: removed "Suppress on demand data update" hack due to bug
* Fixed: needless requests for data updates (should improve the speed of updates in the Strategy Monitor) (found by hlh)

Try out the new Classification Groups feature! Datafeed offers over 90,000 of Stocks (Industry > Supersector > Sector + filter by Country), over 15,000 of Indexes (broken by Country), and over 150,000 Funds (broken down by Type and Country).


P.S. Our Wealth-Lab Wiki hosts the classification files for the provider to download. No Refresh classification option is provided -- we do not plan to refresh the database often.

Thanks Eugene.

Can we get historical index data of Gics industry group of US from B. indexes?

You're welcome.

No, it is not possible because such data does not exist on datafeed's website.

While updating it gives this error and stops updating further:

"Error: Object reference not set to an instance of an object."

Most likely, a symbol's data file has become corrupt. But if you expected a silver bullet after posting this terse error message, sorry but I've come with some bad news for you. The provider works just fine, and if you need a solution, give us something to work with: happens every time or not, on some particular DataSet or not (you might want to perform updates by DataSet to figure it out), does it choke on a certain symbol, at least a fragment of update log showing the error, and the DataSet.

Hi

This morning, the BBFree EOD can't update the following list of European symbols

AAL:LN AI:FP ALV:GR AZN:SS BATS:LN BAYN:GR BG/:LN BMW:GR CA:FP DGE:LN DTE:GR EDF:FP ENEL:IM ENI:IM EOAN:GR FP:FP G:IM GSK:LN GSZ:FP HMB:SS IBE:SM NESN:VX NOK1V:FH NOVN:VX OMX:IND OR:FP RDSA:LN RDSA:NA RDSB:LN RWE:GR SAB:LN SAP:GR TEF:SM TLS1V:FH TLSN:SS TSCO:LN VOD:LN VOW:GR ZURN:VX

Why is this? Has it got anything to do with the fact that 24/11 is a US holiday? I need help...

Have reinstalled provider.

Regards Jon Brewer

Hi,

What does "can't update" mean, exactly? What made you think so (specific error message, last date etc)?

P.S. Reinstalling is useless but at least it's a harmless action.

Eugene,

Sorry, I was unclear. Here is the update log:


Updating DataSet Floyd NormalizeMe ...
Up-to-date symbols: 39, Update required for: 0, New symbols: 0
Symbols already up to date: AAL:LN,AI:FP,ALV:GR,AZN:SS,BATS:LN,BAYN:GR,BG/:LN,BMW:GR,CA:FP,DGE:LN,DTE:GR,EDF:FP,ENEL:IM,ENI:IM,EOAN:GR,
FP:FP,G:IM,GSK:LN,GSZ:FP,HMB:SS,IBE:SM,NESN:VX,NOK1V:FH,NOVN:VX,OMX:IND,OR:FP,RDSA:LN,RDSA:NA,RDSB:LN,RWE:GR,
SAB:LN,SAP:GR,TEF:SM,TLS1V:FH,TLSN:SS,TSCO:LN,VOD:LN,VOW:GR,ZURN:VX,
Update completed (0.88 sec)
----------------


... but last date in chart is still 23/11. The problem is that the provider thinks data is up-to-date, yesterday's data from 24/11 is not there!

Regards Jon Brewer

Okay, got it. There is no problem, this is by design.

Yesterday (11/24) was a U.S. holiday (11/25 is a shortened session). BBFree supports the Market Manager. Since you haven't added these (European) symbols to a separate Market, the Market Manager tool considers them to be "U.S. Equities" symbols (used by default for any symbols not specified in other markets) and will not make data update requests for a holiday date.

Consequently, there are two workarounds:

1. Uncheck "Use this market by default..." for "U.S. Equities" in Market Manager or
2. Add these symbols to a new Market where 11/24 will not be holiday

Repeat update.

Eugene,

These are my current settings:

Market Manager: no market has a check in the checkbox "Use this Market by default..."

I created a new market and imported the symbols from my dataset. This new market has no holidays. All providers have Market Manager support enabled.

It is still not updating...

Updating DataSet Floyd NormalizeMe ...
Up-to-date symbols: 39, Update required for: 0, New symbols: 0
Symbols already up to date: AAL:LN,AI:FP,ALV:GR,AZN:SS,BATS:LN,BAYN:GR,BG/:LN,BMW:GR,CA:FP,DGE:LN,DTE:GR,EDF:FP,ENEL:IM,ENI:IM,EOAN:GR,
FP:FP,G:IM,GSK:LN,GSZ:FP,HMB:SS,IBE:SM,NESN:VX,NOK1V:FH,NOVN:VX,OMX:IND,OR:FP,RDSA:LN,RDSA:NA,RDSB:LN,RWE:GR,
SAB:LN,SAP:GR,TEF:SM,TLS1V:FH,TLSN:SS,TSCO:LN,VOD:LN,VOW:GR,ZURN:VX,
Update completed (0.87 sec)
----------------

Yes, I see it can happen -- not sure why but still looks like a holiday thing.

Workaround is simple: truncate one last bar of this BBFree DataSet using the Data Tool and then repeat update, it will work then.

Eugene, Thanks! That did the trick!

Regards Jon

Is it possible to add an option to select all symbols from a market for the next release ?
At this time, you can the possible to select only subsector by subsector.

Yes, it's possible to select all symbols from a country. I plan to add this in upcoming release.

Thank you very much Eugene !

BBFree EOD stopped updating symbols that start with ".", for example ".TEDSP:IND". Can this be fixed?

I'm using extension v2011.11.1 and WLP v6.3

No, this can't be fixed.

Since it was working, any particular reason why?

Is it because it's technically too complicated, the datafeed won't allow it, or not worth the time?

The symbols that begin with "." is small but there are a few that are timely at this moment such as

.SPN:IND - Spain 10 year spread
.IT10:IND - Italy 10 year spread

Other useful symbols are
.AAIIB2B:IND - AAII bulls to bear ratio
.TEDSP:IND - Ted Spread

Please reconsider.

I agree with Richard1000. If you can fix it, I would be a good things.

I can not fix it, because nothing is broken in the provider. Datafeed thinks it's an invalid ticker, and returns an empty stream. Even though the very same symbol is accepted when you chart it on their website. What happens at their backend is completely beyond my control.

BBFree has been updated to version 2012.05.

Highlights:

* New: revamped Stocks Classification wizard, supports adding entire industries
* Change: refreshed Classification groups (actual state as of April 2012)
* Change: possible to install in Wealth-Lab 6.3+
* Fix: uninstalling BBFree provider will keep Market Manager library

To benefit from additions and changes to the refreshed Classification groups, you have to delete the files already downloaded to your hard drive, manually. We did not plan to update the classification, but in future versions we may consider adding this capability directly to the New DataSet Wizard. For now, all you need to do is:

1. Open this hidden folder (make sure that hidden file/folder visibility was enabled in Windows Control Panel, "Folder Options"):

+ "c:\Users\user\AppData\Roaming\Fidelity Investments\WealthLabDev\1.0.0.0\Data\B.......gProvider\.." (or Pro for WLP users) - Vista, Win 7/8, Server 2008 R2

+ "C:\Documents and Settings\user\Application Data\Fidelity Investments\WealthLabDev\1.0.0.0\Data\B.......gProvider\.." (or Pro for WLP users) - Windows XP, Server 2003

2. Delete 8 plain text files with *.TXT extension. Don't delete any other folder.

This step is completely optional. Already downloaded B. classification groups will continue working with this new provider version.

I have recently downloaded the newest version 2012.05, and I do not see an option to use the BBFree data when I go to create a dataset. I did not have any previous data from BBFree on my machine. Any ideas?

Might not necessarily be a BBFree provider issue; try this suggestion first -- Extension installed in Wealth-Lab Developer doesn't show up after restarting application. If it didn't work, please create a support ticket.

I only have 64 bit installed and still can't see it.

Suggestions?

There is a file in Fidelity Active Trader (x86) called wealthlabpro.dll which is extensions but does this matter? Thanks.

Does the BBFree extension not work w/ 64bit?

I'm 200% sure it works with x64, otherwise you would see a note. Double check that you installed the right one. If necessary, follow up in a different topic to keep this highly specific thread focused.

Known issue: company names are not downloaded (mostly) due to a datafeed change.

To be fixed in next build.

The BBFree provider has been updated to version 2012.11.

Highlights:

* Fix: company name wasn't downloaded due to a datafeed change
* Change: possible to install in Wealth-Lab 6.4+ (requires .NET Framework 4.0)

Classification Groups have been updated on the server. Now they contain:

* 171,633 funds
* 18,345 indexes
* 91,250 stocks

Help

Reinstalling WL and have downloaded BBfree extension which appears to be loaded on WL64Bit. Unfortunately post install BBfree datasets do not appear or does the ability to create new datasets based on bloomberg data.

Sorry w/ the trivialities but I can't figure out what I am doing wrong.

Also sorry about empty posts.. Tried to delete but clearly my internet is on the fritz cause buttons not working.

When you have both x86 and x64 versions of Wealth-Lab installed on the same PC, the last Wealth-Lab edition installed will acquire all newly installed/updated extensions, if downloaded from our website, by default. For example, installing an Extension in x86 version, closing it and starting x64 version makes you believe the Extension is gone. You could find it in the FAQ.

But if DataSets do not appear, this indicates that you are starting Wealth-Lab from a different Windows username. Running Wealth-Lab as admin or as regular user, for instance, will result in having different (or missing) DataSets. That's because configuration and data files are stored separately for each Windows user account.

Hope that helps.

Note to users:

The provider will no longer be returning company names for ambiguous symbols like "AAPL". Currently, it appends garbage to Wealth-Lab charts where company name should be:



Starting from next month's update, an empty string will be returned instead of company name. A recent website change made the data vendor return company name if the symbol contains exchange code (e.g. "AAPL:US").

Chances are that someday they might even stop providing data for symbols like "AAPL" if they're not suffixed with proper exchange code i.e. "AAPL:US". If that ever happens, now you know how to fix it.

The BBFree provider has been updated to version 2013.09.

* Fix: website change broke company names. Now company names are returned only for symbols with exchange codes!

Hi, I can only get data back until 2005. Did I miss a setting somewhere or is there no more data available?

~1500 bars is the maximum you can squeeze from this feed. Pretty good for being free as in beer.

good, just did not want to miss anything.

Last November I noticed that classification groups on the BB website became incomplete (compared to 2012) so I decided to skip scheduled update. Having revisited their website today, I saw that the lists of indices, stocks and funds have gone for good altogether.

Since there's nothing to update, the BBFree classification groups will stay as is forever: actual as of October 2012.

What's new in version 2016.04:

* Fix: malformed data received on rare occasions (updating a huge DataSet) should no longer stop update
* Change: possible to install in Wealth-Lab 6.8+ (requires .NET Framework 4.5)
* Change: (internal) updated user agent version to avoid being recognized as a bot

As reported previously by users, the feed stopped delivering data since early August 2016. In an attempt to fix I had discovered a new limited feed at BB that delivers up to 1 year of daily data and was about to include it when I noticed that the data is Close only. There's no way to get an OHLCV from them. Sadly but the provider has to be decommissioned.

Due to demise of the free data feed we have to take down the BBFree provider from the Extensions section.