Upcoming BNLS improvements

[Skywing]

A new version of BNLS is scheduled to be released within the next few days.  While most of the changes in this version are internal and won't be directly visible to you (such as x64 support, bugfixes, and performance improvements), there are some changes that may be of interest:

- More intelligent version check cache.  Responses that are cached will be lower latency now as they won't have the added delay of waiting to be processed by a dedicated worker thread.  Additionally, cache hit frequency is expected to be improved as there is now a single unified class instead of one per processor.  The impact of this is that some version check operations may be slightly faster than before, and that cached operations will usually complete before an operation that requires actual calculatuions to be performed.  There has never been a guarantee that version checks execute in the order they were requested in, so noone should be adversely affected by this.

- There is a new version check message (BNLS_VERSIONCHECKEX) which combines the version code request and version check request operations into a single request, with support for supplying a user cookie to make it easy to track multiple outstanding version check requests client-side.  Even if you are not going to use multiple outstanding version check requests, it is recommended that you use this feature as it can reduce logon latency and processing logic somewhat (no need to wait for both BNLS_VERSIONCHECK and BNLS_REQUESTVERSIONBYTE to complete, and no extra latency incurred by doing the two messages serially if you were previously doing so).

- Support for detecting a rare condition that causes Battle.net to occasionally improperly reject "NLS" logon requests.  BNLS will notify clients of this condition by sending back a zeroed password proof.  The only recourse in this case is to reconnect to Battle.net, as due to a Battle.net server bug, the logon request would have always been rejected by the server.  There is no way to reliably work around this particular problem without aborting the logon sequence.  The documentation for the BNLS_LOGONPROOF and BNLS_CHANGEPROOF messages has been updated accordingly.  Blizzard has been aware of this Battle.net server bug for over a year, and since it would seem that they have no intention of fixing the problem, this workaround has been implemented in BNLS.  Clients that are not aware of this new BNLS behavior will continue to fail their logon requests in this scenario; the only difference is that new clients can be made aware of a situation in which the logon request would have always failed.
The server "NLS" messages supported by BNLS are not affected by this bug.

- Improved patcher support.  If you have access to direct BNLS to apply a patch MPQ, it should now support all of the new MPQ format additions (in particular, bz2 compression) and so you will no longer have to resort to uploading raw files for some patches.

The protocol specification has been updated to include documentation on the protocol changes.

These enhancements are not yet live on bnls.valhallalegends.com and are currently still in internal testing.  This post will be updated when the new server version is rolled out to production use.

[UserLoser]

Should add some optional cookies to BNLS_LOGONCHALLENGE, BNLS_LOGONPROOF, BNLS_CREATEACCOUNT, BNLS_CHANGECHALLENGE, BNLS_CHANGEPROOF, BNLS_UPGRADECHALLENGE, BNLS_UPGRADEPROOF.

I don't understand BNLS_VERSIONCHECKEX.  It gives you the data you need for say, SID_AUTH_CHECK, but the version code is in SID_AUTH_INFO, so what use is getting the data for both those messages at the same time if you originally need a version code for SID_AUTH_INFO to reach SID_AUTH_CHECK?

[vuther.de]

And I thought out of all that idiling you people did that you'd actually update BNLS.

[Skywing]

Should add some optional cookies to BNLS_LOGONCHALLENGE, BNLS_LOGONPROOF, BNLS_CREATEACCOUNT, BNLS_CHANGECHALLENGE, BNLS_CHANGEPROOF, BNLS_UPGRADECHALLENGE, BNLS_UPGRADEPROOF.

I don't understand BNLS_VERSIONCHECKEX.  It gives you the data you need for say, SID_AUTH_CHECK, but the version code is in SID_AUTH_INFO, so what use is getting the data for both those messages at the same time if you originally need a version code for SID_AUTH_INFO to reach SID_AUTH_CHECK?

You may find it useful for products for which you can use SID_STARTVERSIONING/SID_REPORTVERSION.  IIRC, battle.net only uses the version code in SID_REPORTVERSION and not SID_STARTVERSIONING.  There are other optimization scenarios that you could take, like using a cached value for SID_AUTH_INFO and if BNLS happens to provide a different value, you could update your cache and abandon the Battle.net connection to start over again with the new value.  This would provide a benefit in most cases as the version code does not frequently change.  The primary use for this message is the addition of the cookie field, however.  You can still use BNLS_REQUESTVERSIONBYTE if you don't want to use the old version check system or don't want to use something like the version code caching system I outlined previously.

The "NLS" routines are intentionally not designed to support multiple outstanding requests because they are stateful and rely on internal state created by previous requests.  I may consider adding support for requesting that BNLS provide storage for a reasonable number of internal "NLS" states, similar to how the server-oriented versions of those messages work.

Inner: Attention has previously been focused elsewhere.  These updates have been a long time coming and are something that got lumped in with "port BNLS to x64".  When there were enough pending things to justify spending a nontrivial amount of time on it, I went ahead and took care of them all at once.

[Ringo]

Is there any plans of adding revision support for old clients such as Diablo 1?

Good job so far

[Networks]

Since no one has officially declared it, I will:

Skywing is officially back from WoW and Skywing is officially active, congrats.

[Skywing]

Is there any plans of adding revision support for old clients such as Diablo 1?

Good job so far

This and support for using the Mac version check files is under consideration - however, I'm not sure it's worth the effort (both up front in coding and ongoing maintenance to keep the files up to date each patch) considering how few people use those.

[Ringo]

Hmm true, but maintenance wouldnt really be a issue with older clients, because they rarely patch.
SC shareware for example is still version 1, and i dont think Diablo has patched in years.
It would save me when i logon them and get the "Error BNLS does not support this client, trying local hashing method.. ERROR no hash files located"

I would use them anyway, anyone else?

Keep up the good work!

[Yegg]

As far as I know those games will never again be updated. They are dead. I believe Warcraft II's last update mentioned that it was the final update for that game. And it hasn't been updated since.

[FrOzeN]

I would use them anyway, anyone else?

I'd use it too.

[Hdx]

I would use them anyway, anyone else?

I would/am!

Like Yegg said, the old clients are.. well, dead.
If they are ever patched that I think blizzard would of earned the time (5 mins tops) to update them. Just for the fact that they updated such old games. Even if you had to manually update them.

~-~(HDX)~-~

[Joe[x86]]

You should really make BNLS decentralized. Have trustworthy people such as Spht, Yoni, Skywing, Arta, all host one. I'm sure you can find some way to deliver the server list.

[Skywing]

Realistically, improving service reliablity is not so simple as just throwing up a new server and adding a new DNS entry and calling it done.  The vast majority of service outages have been the direct result of a network problem at some sort.  Additionally, there is no way designed in to the protocol to support the selection of multiple servers or communicate an awareness of up/down to clients.  The most compatible way to do this with existing clients would be through altering which addresses the BNLS hostname resolve to.

So, now you are talking about doing things like setting up a network of highly available DNS mirrors that will be automatically synchronized with zone tranfers or some mechanism, and at disparate physical locations using different upstream providers.  But it's not that simple, because of a BNLS site goes down, you will want to temporarily remove that site from the list of addresses returned on a query for the BNLS hostname or clients will have an (n in k) chance to pick a dead site (where n is the number of down sites and k is the number of total sites) and time out while trying to connect.  This of course should be replicated to all of the nameservers responsible for bnls.valhalhallalegends.com.  To better support something like this, it might be necessary to drop the ttl for that hostname to something fairly low, which would potentially significantly increase load on those nameservers.

So now in addition to setting up this redundant nameserver network, you also need to devise a means to periodically poll all of the BNLS sites to verify connectivity and that the actual BNLS service itself running on that site is up and update nameservers accordingly.

Additionally you will want to provide some means to synchronize configuration information, access control lists, and version check files between all of the BNLS sites.  Some of these use BotNet as their backend database provider, so you need to go and add support for multiple sites into the existing BotNet server, otherwise replace all of the code using BotNet for a database provider with something more suited for multi-site replication.

Even if you decided to replace the parts that use BotNet, though, BNLS has an administrative interface for bot authors available over BotNet that still needs to be supported for use by authors, and you don't want that to go down when one BNLS site goes down either.

There are other concerns as well - will the BNLS authors have the same level of control over site security at these other locations as they do for where BNLS is hosted at now?  Maybe if they were dedicated sites that we paid for, but probably not for just friends running a server in a spare computer in their basement.  Given the kind of sensitive information BNLS has access to, this is not something that users may be comfortable with.

All the while you should keep in mind that things like redunant servers at different geographic locations with different upstream providers are hardly free, and none of us who maintain BNLS make a dime off of the whole thing either.

So you see, it really turns out to not be such a simple thing to design and implement a "high-availability" redundant system that actually provides a good end user experience when sites actually start to go down.

Personally, I'm pretty happy with how well BNLS has done so far.  It services an incredible amount of users per day (last time I checked statistics on it, it was doing approximately one million connections every two and half to three days) and for running on just one server, it's performed admirably with fairly low downtime.

[Mephisto]

One million on Thing's crappy server?  That's impressive. 

[Skywing]

At some point expected to be in the next coming month or so, we will be performing hardware upgrades/replacement for the the server that currently runs BNLS.

During this time I will be reevaluating the need for the various existing BNLS compatibility hacks currently in place to support clients using the old (.91) bnls.valhallalegends.com IP address (or other hardcoded IP addresses), or hostnames other than bnls.valhallalegends.com.

As the official protocol documentation has stated for some time now, you should be using the bnls.valhallalegends.com hostname to access BNLS, and not hardcoded IP addresses, or other hostnames that "happen to" work at this time, such as www.valhallalegends.com or valhallalegends.com.

Please ensure that you are not continuing to use unsupported addresses or hostnames to access BNLS, as depending on the situation, these may not remain valid indefinitely.