Hashtable insert failed error when updating data by IQFeed and Yahoo
Author: superticker
Creation Date: 1/12/2021 3:57 PM
profile picture

superticker

#1
I just upgraded from 100Mbps to Gigabit Ethernet, and it all works great. Wonderful speed. Very happy! However,...

... I'm now having some "simultaneous threading problems" with two WLD data providers. For IQFeed, all on-demand updates fail with the green error message below. Chart updates from the cache (for Daily, Weekly, and Monthly bars) work just fine.

For IQFeed static daily-bar downloads, the first several stocks fail. After that, everything seems to work fine. I'm assuming this is a problem with the IQFeed Client and not the WLD IQFeed Provider. Can someone please confirm this?

Next, how should I adjust the IQFeed Client (or Provider) to fix these problems?




Static updates for the Yahoo! provider are totally broken. Nothing works there--see screenshot below. I tried reducing the number of Yahoo! Internet threads from 10 to 1, but that didn't help. How can I best fix this problem?



Finally, these error messages from both IQFeed and Yahoo! look very similar. Exactly where in my system does this "Hashable insert" error occur? I'm thinking this has more to do with .NET asynchronous thread creation, but I'm only guessing.
profile picture

Eugene

#2
Looks like it's a repercussion of your problem: IQFeed error popup: Hashtable insert failed with GetExternalSymbol. By the way, neither Yahoo nor IQFeed provider uses Hashtable. This may have to do with obfuscation as that's what both have in common.
profile picture

superticker

#3
QUOTE:
IQFeed error popup: Hashtable insert failed with GetExternalSymbol.
So you're saying there's a race condition in the WLD core code that's made 100% worse by the Gigabit Ethernet. So how do I best fix this?

What about the static daily-bar updates? Why is there a "Hashtable insert failed" on the first 7 stocks (although all following updates work okay)? See IQFeed static update screenshot in Post# 1. Apparently, static IQFeed daily updates are using the hashtable too.

There's more. When using a plain Chart window, I get a "No Data Available" on all IQFeed on-demand updates for 1-, 3-, 10-, 15-, 30-, and 60-minute charts. For the 5-minute scale, the plain Chart works about 50% of the time; otherwise, I get "No Data Available". There is no "Hashtable insert failed" though for the plain Chart window.

So how do I make the plain Chart window work again for on-demand IQFeed updates?

What about static Yahoo! updates. None of them are working. See second screenshot grouping in Post# 1. And it looks like static Yahoo! updates use the hashtable for everything. Is the static Yahoo! provider calling GetExternalSymbol? (I can't imagine why?)
profile picture

Eugene

#4
Sorry to rain on your parade but this apparently not a race condition, network or CPU cache issue. I'm saying this may be a defect in our obfuscation of the DLLs. That's my vision.
profile picture

superticker

#5
If it's not a race condition, I'm not sure why this problem would be Ethernet speed dependent. It worked fine (99%) under 100Mbps, but not well at all under 1Gbps. In both cases, the code base is exactly the same. Streaming works fine in all cases.

QUOTE:
... a defect in our obfuscation of the DLLs.
Hmm. So you're saying the encryption of the DLLs (like for the static providers) is employing the hashtable to decrypt? Once it's JITted in memory as native code, shouldn't it be okay? That might generate a single error message (initial JIT operation), but it would run fine after that (as cached native object code). But that's not happening; the errors continue to occur. Clearly, I don't understand.

Thanks for the feedback. Sounds like you're saying there needs to be a patch. I just want to get my WL client working again.

Does the obfuscation of the WLD6 code even matter since you're leaving that code base behind? (rhetorical)
profile picture

Eugene

#6
If you create a support ticket I could run a custom build for you to try.
This website uses cookies to improve your experience. We'll assume you're ok with that, but you can opt-out if you wish (Read more).