Experimentalcraft

Anyone who frequents the Icy Veins Forums (the Shaman section, at least) will know that I am a huge proponent of planned cooldown rotations. I never shut up about them. I’m of the opinion that “raid cooldown choerography” is one of the most important things to distinguish yourself as a good Resto Shaman. I mean, we have so damn many of them that you’d be stupid not to use them. With that in mind, an interesting (exhaustively well made) post about raid cooldowns caught my eye a week or so ago. The general gist (for Shaman) was that;

  1. Spirit Link Totem is awesome, use Spirit Link Totem.
  2. Ascendance outperforms Healing Tide Totem when used optimally
  3. Ancestral Guidance is insane. Literally nuts-in-the-salad ridiculous.

While 1. came as no surprise to me, 2. was interesting (I had my suspicions) and 3. was a downright shock! I was initially sceptical (coming up with several possible reasons for a misleading result) but the research was genuinely well executed and the only question I had un-answered was does it do that well in reality? Good question, that one. Let’s start by showing off the numbers;

Spell Healing (“total”) Healing (rotation subtracted) Healing (with EM)
Healing Tide Totem 2.5M 2.5M ???
Ascendance 6.4M 3.2M 4.3M
Ancestral Guidance 7.2M 4.7M 6.4M

That is a BIG difference in healing! (See the original post for caveats on the numbers; rotation required, gear assumptions, calculations, etc. The ??? signifies that it’s somewhat challenging to predict what HTT will do with Haste stacking.)

So: what happens in reality? Well I decided to take my current Haste build for testing and use Ancestral Guidance on Megaera to see how it stacked up. I tried my best to use Ancestral Guidance and Ascendance both optimally on two Rampages each, here is the log. You can see massive peaks in the healing where each Rampage happened. As a side observation, all the mana was spent in the low-HPS phases rather than the peak phases.

To properly interpret this data let’s first make sense of the cooldown rotation that we had set up. Since what we got was sliiightly different to what we planned, this is as close as I can tell what happened;

  1. AG + Fire Elemental + Power Word: Barrier
  2. Ascendance Macro
  3. Spirit Link Totem + Earth Elemental + Tranquillity
  4. Ancestral Guidance used with Elemental Mastery + PWB
  5. Fire Elemental + Devotion Aura + Tranquillity
  6. Ascendance Macro (Ancestral Guidance on cooldown)

So we see six peaks in the Healing graph. What I notice immediately is that the two where I used neither Ascendance or AG had no real peak for me personally (as of right now it’s refusing to plot a whole-team graph). In contrast, one of my peaks when using Ascendance is almost 400k HPS. Holy. Fucking. Damn.

Anyway, looking at the more sane table of healing values we find that overall Ascendance and AG did similar numbers of effective healing. AG did 5.8M healing to Ascendance’s 5.66M and AG overhealed by 52% compared to Ascendance’s 75%. You read that right: seventy five percent overhealing. Insanity. Taking account of the fact that both Ascendances used EM but one AG did not, I calculated the (approximate) distribution of healing done by spell, not taking into account health-levels;

Spell Healing (Million) Total Healing (Million)
AG 1 3.45 3.45
Asc 1 1.30 5.20
AG 2 2.35 4.70
Asc 2 4.36 17.44
AG 3 2.35 3.45

So as predicted Ancestral Guidance does appear to outperform Ascendance under similar conditions. However this is only really a first-order analysis; in reality the situations on each Rampage were much different. As Dedralie pointed out in the comments of her article, the mean health percentage of the raid goes down on each successive Rampage, so there is always an effect where Mastery is pushing Ascendance higher than AG (since I used it on average later in the fight). In addition, the overhealing says a lot about each cooldown. Neglecting the effect of mean health percentages for now, consider the potential reasons for each mechanic overhealing;

  1. AG is a very smart heal – it will only really overheal when the players on lowest health in the stack are at higher health than 60% of the heal you just cast.
  2. Ascendance is NOT a smart heal; each raid member gets 10% of the healing you do each second. If one person is more damaged, they get less overhealing. The effectiveness then depends on how spread out the raid damage is (one of the reasons that you REALLY want to use SLT with Ascendance).
  3. Ascendance has a longer duration; considering that the other healers will be working hard to top the raid, it’s more likely to have time where the raid is at overall high HP. In that sense, it is MUCH more likely to overheal than AG.

Now this all so far points to Ascendance being the biggest overhealer (as observed), but then in addition consider that at higher HP, all of the above mechanics become more likely. That means that using cooldowns early in a fight will almost always cause more overhealing than when they are used late. So; why would you want to use cooldowns early? Simple answer: it does more healing overall, even if it overheals more. Consider the rotation I used in comparison to just using each once, towards the end of the fight. Even in an optimistic case of low (~25%) overhealing on only one use of each cooldown, using it only once produces roughly the same overall healing as the realistic case – in reality you will see a significant drop in overall healing.

When we take a look at the total healing done by each spell inclusive of overhealing, which should be a more close test of the hypothesis, we find that;

Effective Healing (M) Overheal (fractional) Total Healing (M)
AG 5.8 0.5 11.6
Asc 5.66 0.75 22.64

I ran these numbers through the same kind of analysis which I used for the total healing numbers. This is done by calculating the ratio of predicted healing values between each cooldown with and without Elemental Mastery active (taken from comments in the post mentioned earlier) and dividing the healing from each spell up appropriately. This method was also used to account for the increase in healing under the effect of Heroism. The numbers are shown in the graph below;

Observed values from the WoL table corrected using a simple estimation technique.

Observed values from the WoL table corrected using a simple estimation technique.

We can see that the predictions for healing values for Ancestral Guidance are reasonably accurate, with a very small difference between the predicted values (from the post mentioned earlier), which is an encouraging sign. More interesting is that Ascendance appears to have healed for a massive amount more than predicted! This is a significant discovery because it implies that there is a flaw in the way that we formulate the healing that Ascendance can do for us.

At this point it occurred to me that you can do another interesting thing using World of Logs; you can select a subset of time in a fight and re-analyse your data for just that time period. Normally I would say this isn’t that useful apart from for e-peen measuring, but now it represents a method of measuring precisely how much each cooldown healed (to 7 significant figures, no less!). It was somewhat laborious to get the data, but below you will see the actual accurate amount which each use of each cooldown healed.

Spell Healing (M) Total Heal (M)
AG 1 1.43 3.40
Asc 1 2.77 9.55
AG 2 2.3 4.51
Asc 2 2.89 13.14
AG 3 2.1 4.38

I went as far as to calculate how much of a difference that amounts to. Below, you can see the results – we have clearly systematically underestimated Ascendance as a raid cooldown in terms of raw healing output.

Accurate observed healing values found using World of Logs' amazing time domain selection tool.

Accurate observed healing values found using World of Logs’ amazing time domain selection tool.

You can also see from the graph above that the esitmates I made to get to the previous graph are somewhat off, but are nevertheless a good approximation of the real data. Importantly, my approximations over-estimated the skew in the distribution of healing by Ascendance. This is, I believe, indicative of the effects of mean health percentages in the raid. That is to say, I think that the average healthbars were different on each use of Ascendance and that this compensated for the difference due to Heroism on Ascendance 2. Either way, we can still clearly see how ridiculously we have under-estimated the power of Ascendance for the raid.

You can also see that the usage of Ancestral Guidance was somewhat below the predicted value. I would say that this is partly because the average healthbars weren’t at the ~40% assumption that was made in predicting the original values. In addition, I would say that my gear is somewhat below the gear assumed in the original post, AND I’d say that my usage was somewhat sub-optimal. With all this in mind, I conclude that it is entirely possible to use Ancestral Guidance to the amount that the theorycraft has previously predicted. The potential still does not meet Ascendance, even if used optimally.

Remember that in many cases you will not be able to gain the advantage of a fully stacked raid – in this case Ancestral Guidance will not be optimized. In this case, or in the case where you have to do a lot of movement, Healing Tide Totem will still be your best healing cooldown. However, if you can get that cooldown optimized, it will rock your world!

To finish this post, I’d like to thank Dayani (the original poster of the theorycraft) for a really awesome chance to do what is essentially science, but for WoW. I loved it! I really think that in terms of Healing, we do a lot of Theorycraft (i.e. formulation) and not a lot of “Experimentalcraft” (i.e. proper testing of the theories). I always feel that this is where the healing Theorycraft community falls down a bit, and I’d like to see more rigorous testing of hypotheses out there.

Advertisements

About stoove

A physicist, researcher, and gamesman. Likes to think about the mathematics and mechanics behind all sorts of different things, and writing up the thoughts for you to read. A competent programmer, enjoys public speaking and mechanical keyboards. Has opinions which might even change from time to time.
This entry was posted in World of Warcraft and tagged , , , , , , , , , , , , , , , , . Bookmark the permalink.

10 Responses to Experimentalcraft

  1. Dedralie says:

    Simply awesome 🙂

    I’m quite surprised by your Ascendance results and I’m going to look over my log from last week where we underhealed Heroic Megaera 25 to see if a similar thing happened for me. Of course there’s an additional mechanic on that fight that mucks up healing analysis significantly, so it’s going to be much harder to tell, but I don’t have access to a normal mode Megaera with a Shaman healer.

    Theorycraft vs Experimentalcraft is an interesting distinction and it’s something that, I agree, has always been lacking. I try to always test everything in-game somehow (I have been using AG instead of HTT for months now in advance of writing that article), because I recognise that spreadsheets don’t match behaviour for healers; it’s not like DPS where if you perform optimally you do the most healing. If your raid performs optimally there’s less healing to do, after all! But I’m a huge fan of applying science techniques to the game – most definitely including experimental work, I certainly did a lot for my article on Spiritual Innervation! – and I’m all like touched and honoured and whatnot that my li’l raid cooldowns article inspired this outcome 🙂

  2. stoove says:

    I’ll be honest I was so surprised by the result that I went and checked several times that the overhealing stat on WoL isn’t included in the Total Healing number. It appears not to be. I’ll have a look at the logs again next time I clear it (should be this week (TM)).

    I think that one area where we can push Experimentalcraft to the fore more often is by just discussing explicitly how we found certain things and the methodology which we used to do it. Many articles I’ve read have actually done *some* experimentalcraft but have only implied it / skimped on details. This is a real shame, since we could easily get more information out of the game. As an example, I like to recall the discussion I had with Binkenstein et al. about Haste interaction with HTT. I was incredibly frustrated at the time that people seemed to think that three datapoints was enough to say “well this is useless, we’re defeated”. Solving that mystery is still something I’d like to do.

    Honestly, I’d love to be able to test more of your predictions about numbers in 10’s. Unfortunately I play Shaman For Life, so someone else is going to have to find the data =]

    • Dedralie says:

      I thought the Haste/Totems thing was settled to be some sort of bizarre server-side lag on despawning the Totems that would occasionally prolong their duration long enough to get you an extra tick despite not having enough Haste to normally get it? I remember there being some experimental evidence showing that it happened at a lower Haste rating for Bink, whose server lag issues are more pronounced thanks to New Zealandery, than for Therya, in Europe, whose ping is better and thus server lag is less prominent. I am not technically knowledgeable enough to know if that is even remotely possible, but it made enough sense to me when I read it (plus it jived with my experience of having up to 0.5-0.7 extra seconds to recall HST and get my mana back).

      I play a Druid in 10s and have a Paladin and occasional Disc Priest co-healer, so I can access some information about their CDs (although my Druid’s gear is far below what my models are using, so there’ll be obvious discrepancies). Working on a Monk now, too. So really that only leaves Holy Priest for data gathering needs! (I have a Priest myself, but I don’t raid on her, as I just can’t find enough groups to support it.)

      • stoove says:

        Well with regards to Haste the evidence we’ve seen so far is… inconclusive. There has been a nice attempt, but the datasets gathered were woefully small for something of this nature. One of the competing hypotheses (mine) was that the ticks were probabilistic in the region of a Haste breakpoint to make the transition between them smooth.

        While my hypothesis is comparatively unlikely, there isn’t enough evidence gathered to actually rule it out. We know for sure that there is An Effect, but we don’t really know why it happens. I think that it’s likely that the latency hypothesis is correct (and even if it isn’t there’s not much likely to change), but it’s interesting to find out for sure just in case we could exploit it.

        It would be nice to see some data from other healers, I might have a go at doing another couple of comparisons. I also have a Paladin and a Druid as co-healers so I’ll see what I can get from them. I don’t think that lower ilvls are such a massive problem – expect a small effect, but as I showed above you can still get close to the optimum for AG with a lower ilvl.

      • Dedralie says:

        I don’t think your theory is out of the realm of plausibility, just that it’d be weird since no other haste breakpoints work that way, and if there’s one thing I’ve come to expect, it’s that perfectly cromulent mechanics would be re-used rather than re-worked 😉

      • stoove says:

        Mmm yes I agree, “plausible” but not “likely” so to speak. Honestly, I really wish that Blizzard had thought up something rather cool like my hypothesis and we’d just not noticed. That would make my week ^__^

  3. Dedralie says:

    Well it’s a bit late for this now, haha, but I thought the response and analysis of recent changes to resto shaman healing on the PTR would help explain the Ascendance anomalies compared to what I had calculated in my blog post, and why it shows up as so much additional overhealing.

    As we had already discussed, my post glossed over pre-existing Riptides and Earthliving Weapons, which both get copied by Ascendance. I didn’t have a really great model for determining how many people in the average raid would be under the effects of Earthliving (especially since I wanted to keep it simple with one calculation instead of a separate entry for 10- and 25-player raids). This probably isn’t going to be a huge swing. It certainly isn’t responsible for the ridiculous disparity between what I calculated and what your logs showed, but it is an oversimplification that I really ought to fix 😛

    Much more importantly is that all diminishing-returns (DR) based healing (e.g. any spell saying that “healing effectiveness diminishes for each player beyond 6 within the area”) and their interactions with pets. At the start of this expansion, pets were causing troubles for these DR heals, because their presence in the area of effect was diminishing the healing done to the actual targets we care about – the players. For example, if you had 10 players and 2 pets in your Healing Rain, the game would diminish the value of your Healing Rain ticks as if there were 12 targets, and thus you’d have ticks that were (6/12)=50% as strong as their maximum value, and 2 of those were wasted each time on pets who we didn’t care about.

    At some point along the way – maybe 5.2, maybe 5.3, I really am not sure – this behaviour got changed, so that pets would no longer eat limited-target smart heals if there were any injured players at all, and so that DR type heals would still apply to the pets, but not count the pets against your DR total. After this change, all 12 targets in the example above were receiving (6/10)=60% value ticks. This caused our spells to actually generate MORE healing than we expected, but since pets (especially short-term summoned pets like Voodoo Gnomes and Wild Imps) do not take much damage in an encounter, the extra healing was mostly overheal, and nobody really cared.

    Then we press Ascendance. All the effective healing that is being dealt gets copied and redistributed over the raid – even that healing to the pets. Sure, it’s overhealing on the dumb pets, but it can turn into effective healing on the raid. So those two extra 60% ticks were another 12% healing on raid members under Ascendance.

    But wait, there’s more! Ascendance is exactly like a DR type heal, and is treated as such by the game. And pets do not reduce the value of DR heals, but are still affected by them. So the pets themselves were also getting the ticks from each other.

    With made-up numbers to illustrate:

    Healing Rain’s tooltip states it will heal for 16667 each time it ticks. With 10 targets in the healing rain, that means each of them gets healed for 10,000 each time it ticks. Each pet you add to the pile adds another 10,000 healing per tick. So with 10 players and 2 pets in the pile, your Healing Rain is healing for 120,000 every ‘pulse’ – a 20% increase.

    Now you pop Ascendance, and each of the 12 10k ticks is getting distributed evenly amongst the raid. Since there are 10 targets who ‘count’ for DR purposes, each player – and each pet – is getting a Restorative Mist heal of 1k from each other player – and each pet. That’s 12*12*1k = 144,000 healing, which is a 44% increase over what really SHOULD be happening if pets didn’t get healed at all by AoE heals, and a 20% increase again over what our pet-boosted Healing Rain is doing already. So Ascendance, which should be a 100% healing boost, was becoming a 120% healing boost thanks to the presence of the two pets (or if you want to say that pets never should have counted for AoE heals anyway, then it was a 144% healing boost).

    Now, of course, the more pets you add the more ridiculous this becomes. In a 25-player raid with 3 warlocks, a bunch of melee with the Voodoo Gnome-summoning trinket, Shamans with elementals, hunters with their permanent and temporary pets, Mindbenders, etc., the extra pet healing really adds up to a lot. (Note that this was part of the exploit that was leading to some Resto Shamans getting crazybananas healing on H Tsulong, back when Ascendance and Ancestral Guidance fed off one another, and Shamans + a snake trap could heal Tsulong to full within a minute of reaching day phase.)

    While again, the only healing we cared about is the extra 20% to the raid members themselves, the pet healing is a huge part of the ridiculously high overhealing we were seeing from our Ascendance in these logs.

    In 5.4, this is being changed slightly. Healing Rain is becoming a hard-target-capped healing spell, meaning that it simply will not heal pets so long as there are at lest 6 valid player targets in its area of effect. This means we lose the two extra 10k HR ticks from my above example.

    However, as of right now on the PTR – and I’m going to try to write up a bug post about this, because I’m not sure it should be working this way – pets are still eligible for Ascendance healing but do not count against its DR. I was just standing all by myself and wondering if my 4pc t16 set bonus summoned Spirit Champion was eligible for Ascendance healing, and here’s what I saw:

    > Your Healing Surge heals you 89101 Nature.
    > Your Restorative Mists heals you 89101 Nature.
    > Your Restorative Mists heals Spirit Champion 89101 Nature.

    Now, it remains to be seen whether this behaviour holds up in raids, and I’m going to pour through my logs to see if it does. Extrapolating it to a 10-player raid with 2 pets and using Ascendance with Healing Rain only, though, here’s what we’ll see:

    Each HR pulse generates 6 ticks of 16,667 healing, for a total of 100,000 healing per pulse.
    Ascendance copies that 100,000 healing and divides it 10 ways, for an extra 10,000 healing per player, and also heals 2 pets for 10,000.

    So we still get 120,000 healing out of Ascendance. However, we are no longer “cheating” and getting an unexpected amount of healing splashing onto players from the pets, so all of the extra 20% is pretty much overheal. We should see the effective healing on our raid members go down by 20%, and our pet-related overheal also go down by about 20%. (The actual overheal may still be pretty high, of course, but that will vary depending on what other healing cooldowns are being used at the time and how much damage the raid is taking, etc.)

    Now, this is definitely a nerf to the observed throughput of Ascendance, although my belief is that it will mostly come out of our overhealing. However, you’ll note if you go back and look at my post that Ascendance still performs very favorably to other cooldowns that other classes have, and I wasn’t counting any pet shenanigans at all in my analysis! So I don’t think this is worth freaking out about or lamenting. The developers have said they do not think it is good design for random-proc pets to be contributing so much to our healing, and I, for one, whole-heartedly agree.

  4. Ascendance being nerfed by removing the pet trickery is fine with me too.

    How about AG’s reduction in throughput…
    Tiberria on the MMO forums claims a 30% loss with the HR changes. (I know he tends to interpret data on the extreme ends)

    Won’t this make taking Rushing Streams an easier choice unless you absolutely need another burst cd? I raid 10 man with only 1 melee so RS was going to be my default choice regardless.

    Blizzard wanted the L75 talents to be a choice. My fear is AG won’t be used much in 10 mans and perhaps 25’s too considering how strong RS currently is. Let’s not even get into how they bungled Conductivity. They had a chance to give us 3 competitive options akin to L90 priest talents.

    Should we start advocating for an AG buff to have at least 2 viable options?

  5. Dedralie says:

    I’m not really sure I buy the 30% reduction in throughput, but basically, my calculations of AG didn’t include the pet shenanigans either, and it still came out as more powerful (and more bursty) than Ascendance. The change to Healing Rain shouldn’t affect the numbers I’ve calculated at all. I’d still use it in situations where your raid can stack and will need a burst throughput CD, and in SoO, those are going to be:

    Several times in the Fallen Protectors encounter, particularly Sun’s Desperate Measures;
    Phase 2 of Norushen;
    Phase 2 of Sha of Pride;
    Phase 2 of Galakras;
    If your raid uses a stack-and-move-together strat for Dark Shaman, it’ll be pretty good there too;
    Phase 2 of Malkorok, which occurs every 2min;
    Depending on strat, Thok the Bloodthirsty;
    Phase 1 of Garrosh Hellscream (but admittedly, I haven’t seen enough of this fight to know if phases 2 and 3 will lead us to ultimately prefer Rushing Streams over AG).

    It’s also a possibility on Immerseus for more healing throughput to healer-slimes, and in Norushen, you can use it while you do your healer trial to help you keep your NPCs topped off more easily so that they do more damage and complete the trial sooner, and still have it back when you need it in phase 2.

    Now, don’t get me wrong, Rushing Streams is amazing too. Especially since I love spreading Glyph of Healing Stream Totem over the raid (and this is, IMO, the real benefit of our t15 2pc bonus atm). But even discounting Dark Shaman and Thok the Bloodthirsty – since the usefulness of AG there depends on whether your raid wants to stack when it is not necessary – AG will be useful on at least 5 of the 14 fights. And to keep things in perspective, it was useful in one fight in t14 over HTT (Tsulong) and in 3-4 fights in t15 (Megaera, Iron Qon, Twin Consorts/kinda, only on heroic, and Ra-den).

    As you note, the real issue here is the inability to make Conductivity attractive to Resto Shaman. (I really think it will end up being unattractive to Elemental as well, and with the nerf to Glyph of Healing Storm, most Enhance will want to stick with AG. In fact, AG is kind of the only talent on the tier that makes sense for the DPS specs, whose Healing Stream Totem is not very powerful, and who aren’t likely to be running the Glyph of Healing Stream.) If we had 5-8 fights for AG, and 3-4 fights for Conductivity, and the rest were Rushing Streams fights, that would feel pretty good.

    I haven’t done math yet – these are observations from testing the raid fights and feeling out when AG is useful. Kinda waiting for the ‘numbers pass’ to die down before I do too much. But rest assured I’ll cover this topic in depth with math and graphs and logs in the week or so before 5.4 launches 🙂

  6. Pingback: To Craft a Sim – Part 1 | UNconstant

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s