Rendering Status: What's (NOT) Been Happening, and How We've Tried to Fix It.
Hello friends! Thank for patience of you. Have be very busy with animate, so no many talk have do. It have be VERY difficult get GPUs for content make, so I do hire of friend in other country to get. Is long, so you should popcorn get. If not want read full story, picture attach is TLDR
Okie, now to technology get friend talk...
---------------------------------------------------------------------------------------
Hey there guys, DT's procurement (and also story translating) guy here. I hope you like disaster stories.
DT asked me to write up on where we're standing with equipment and rendering resource. If you're not interested in knowing, then don't bother reading this bit, as there's nothing horny here. Unless you find supply chain managment of electronic devices arousing. I don't kinkshame.
The following chronicles the absolutly unadulterated and terrifically poor shitshow we've had trying to get cards capable of rendering the last (and most intensive) scene in the animation. Put VERY simply, the current GTX 1080s DT has available to them can render about 1 frame every 15-30 minutes: Good for a quick 2 minute animation, but terrible for the 5 minute monsters that have multiple scene changes, angles, and all that good stuff.
Put simply; The 3rd protogen x synth animation was made on the understanding that they'd have some truly epic equipment to render it on, if the benchmarks were legit.
In October, NVIDIA released a new graphics card called the RTX 3080 & 3090. Every 3080 they got would more than double the previous rendering speed of their rig, thus making rendering 2x faster. Every 3090 would 3x it (though they are slightly less cost efficient). It therefore made sense to try and pre-order as many 3070 cards as possible, as we knew supply would be limited and if we didn't buy early, we wouldn't be getting any.
In order to make sure that we'd get at least SOME cards early on, we attempted to preorder six inidivdual cards, consisting of three ASUS TUF GAMING and three MSI GAMING X TRIO, from multiple different suppliers in multiple different coutries. Total initial investment in cards: about $5000
However, these were not the only costs involved, because you can't run GPUs without the surrounding hardware. To go alongside these orders, we also attempted to procure
- Three 1000W PSUs of ANY brand
- Three kits of 32GB of RAM
- One Ryzen 5900 series and two 5600 series AMD CPUs
- Motherboards compatable with the above
- 3 of the cheapest cases that could fit 2x RTX 3080s
All in all, approximately $9000 was initially spent on JUST trying to get eqipment once you include shipping and taxes. This investment exceeded two months of patron earnings (in October, about 10% of patrons had declined payments, plus patron, paypal, and currency exchange fees reduced the total income last month to about $3100, for context). There's then also the cost of purchasing textures, music, and other misc assets for the animation. Some cash was taken from money saved for such an occasion on order to cover this, whilst some additional cash was fronted on credit.
This plan would have worked rather well if not for the fact that so far, NONE of the equipment we actually needed in the first place has arrived; DT currently has no more processing power available to them than before they spent nearly $9,000 on it. Here's everything that went wrong:
- In the case of one retailer, no correspondence was received for two weeks, after which it was noted that the amount charged was higher than the price indicated at check-out. Things are currently stuck in a dispute with the advice from them to simply cancel the order and reorder it - something we can't do as they have closed re-orders.
- Another retailer indicated that there would be no guarentee that they would be able to provide a PSU until after Christmas. We asked them to remove this from the order (that also contained 2 GPUs). Instead, they instead cancelled the entire order. We were unable to reorder the GPUs.
- At one point, one retailer offered the option of upgrading two of our out-of-stock 3080s to two in-stock 3090s for an additional $2400. Only after we did this transaction did we discover that the 3090 had gone out of stock in the 1 hour it took to agree to this, with an unknown restock date. After waiting for a month, we requested a refund.
- Currently, only one retailer has been able to actually provide estimates on when they will be delivering, though it must be noted that these have been sparce and on several occasions they seem to have forgotten we ordered TWO cards with them. The current estimate they have is that they will be able to provide the GPUs "before Christmas"
After we graphed out the position in the queue they provided us, it's unlikly they'll arrive before March. - On Thursday, another supplier listed multiple units of RTX3090s available. We were able to procure 2 of these, for delivery on Friday (Today), for an inflated price of about £4000 (~$5000). However, this evening we received a message indicating that there was no stock available. This happened two minutes after their customer service line closed, and we will be unable to contact them until Monday. It's almost like they intended this.
- The availability of PSUs is even less clear. (Yes, there's a high-performance PSU shortage, because due to COVID everyone's working from home, and due to the RTX 30xx series, everyone is buying high power PSUs)
- We gave up trying to find any current-gen Ryzen CPUs, which tells you everything you need to know about THAT
- IN ADDITION, I also tried to contact NVIDIA directly before and after the launch, both on social media and via email. Many other creators with a similar (albeit not adult) following appear to have been provided cards on release date by NVIDIA at RRP, and we attempted to get in on this. NVIDIA has not responded to correspondence. This story is repeated with GIGABYTE and MSI.
Without wishing to sound crass, it has been an utter shitshow. We have spent almost $9000 and in return received nothing but bad managment, poor information and bad customer service. SOME of this money has been refunded, however in other cases we are left waiting for either orders or refunds.
Whilst some of the refunds provided some money so that dreamer could access a render farm as a last ditch resort, this has also had problems: The render farm that provides the right performance at an accessable price without banning NSFW stuff has had zero availablity every time DT has tried submitting a job (presumably because everyone needs it due to GPU shortages)
All cancelled/refunded/stalled orders accounted for, we currently have:
- 1 x RTX 3080 ASUS TUF GAMING, due some time in March
- 1 x RTX 3080 MSI GAMING X TRIO, due some time in June
- 2 x RTX 3090 MSI VENTUS 3X OC, unknown due date, POSSIBLY next week
- $5k worth of pending refunds
- Two motherboards and two kits of 32GB RAM.
Once the refunds are processed, we should be able to find a render farm that can help with the initial stuff waiting to be rendered. Depending on if the 3090s are going to be available, either they or the 3080s will also be refunded, though we're waiting on the supplier's customer service to reopen.
On a personal note, I am frustrated with every single person involved in this, up to and including myself. NVIDIA & AMD should have ensured sufficient supply, I should have tried getting PSUs earlier, and the retailers could have provided clearer, faster or just downright HONEST communication. I know this is frustrating DT too, because even though they're working their socks off, the poor behaviour on the part of other people has made it look like they're not doing anything, which is very much not the case.
I'm not going to apologise on behalf of our suppliers and their associates, as they deserve your scorn for poor managment. I will, however, apologise for not expecting this and not helping DT plan better.
DT also wanted me to remind you in the most eloquent fashion that I can that if you're ever unsatisfied with the rewards on the patron, you shouldn't feel tentative or worried about unsubscribing. They're not in this for the money, and as we've seen here, sometimes there's stuff that you can't get no matter how much money you're willing to throw at the problem. Your support helps make the animations you enjoy faster, but nobody - especially not dreamer - is going to judge you for leaving, given how much people need to be saving to look after themselves in this economy.
The pair of us seek your understanding and compassion in how we've got caught up in this very exciting but also very turbulant moment of progress, and look forward to providing the highest quality of animations the furry fandom has seen, regardless of your financial support.
Cheers, M