lordaardvarksfm
patreon
Progress Report - February 1, 2023
🕑 Added 2023-02-01 20:56:04 +0000 UTCHello, everyone.
Let's pretend that my making this week's Progress Report on February 1 was totally planned, and not just a happy coincidence resulting from the fact I haven't really had anything noteworthy to report this week.
Also, yep. You guessed it. The post image is another bait and switch.
Artemis and Medusa aren't the main focus of this post. Though they are part of what I got done last week. Read on and you'll get to them eventually.
The Auto-Releaser - Dependencies
The majority of the week has been spent working on the auto-release system. Though at this point it's really more of a full-on content management system I'm writing.
It's taking a bit longer than I had originally anticipated, as I keep throwing solutions to problems I've previously had to Just Deal Withβ’οΈ. The big one I've recently implemented is the dependency system:
Here's a brief rundown of what it is - and more importantly, why it matters to you the end-users:
A lot of my assets share things. The most obvious examples is the fact that all of my character models use the same IK rig, the DazV5-5 IK Rig. But there are more subtle things they share too. For example, all of my character models are designed explicitly to be used with the Outfit Loader, the Model Favoriter, and the DazV5 Bodygrouper. And more technically, all of my DazV5-5 character models use shared body textures, to reduce how much memory they take up in SFM - allowing for more characters to be spawned in scene, making the large orgy scenes I love so much to be feasible.
In all of these cases, these are the same files that are packaged for every individual model. For every character model you download, you have to download these same files over and over again. And every time you install them, you have to overwrite the existing files.
In isolation, that's fine. It's a bit redundant, taking up space and slowing downloads with files the downloader may already have, but it guarantees that any character model is complete, and will work out of the box if it's that person's very first character model ever downloaded.
However, imagine this situation:
> I release a character - say Femshep. Like everyone else, she uses shared body textures and scripts. The version of these textures and scripts she is packaged with is v1.
> A few months later, I make changes to the body textures and the scripts, moving them from v1 to v2. These are marked improvements for all character models, a straight upgrade that only improves all models that use them, past and present.
> I release another character - say Miranda. She uses the same shared body textures and scripts as Femshep. But now, she comes packaged with the new and improved v2 textures and scripts.
Now with this situation in mind, imagine two theoretical people downloading Femshep and Miranda. The difference is the order they download and install them in.
> Person 1 downloads Femshep first, and then Miranda. In this case, everything is fine. When they install Femshep, there are no conflicts, everything just installs. When they download Miranda, they are prompted to overwrite the shared textures and scripts. This is fine, because the version they currently have installed come from Femshep, and are v1. The ones they are overwriting with come from Miranda, which are v2. So they overwrite the old v1 files with the new v2 files, and all is well.
> Person 2, however, downloads Miranda first, and then Femshep. When they install Miranda, there are no conflicts, everything just installs. But when they then install Femshep, we have a problem. As before, they are prompted to overwrite the shared textures and scripts. But in this case, they should not overwrite, because the textures and scripts they already have come for Miranda, and are v2. But the ones they are prompted to overwrite with come from Femshep, and are the outdated v1 files. If they overwrite, then they're not only downgrading Miranda, but are keeping Femshep from being improved with the newer v2 textures and scripts Miranda comes with.
I think you can see the problem.
The solution I have come up with, then, is dependency. Rather than having both Femshep and Miranda have the same textures and scripts included in their individual downloads, have those shared textures and scripts be a third, separate download. And then just have Femshep and Miranda depend on that third download.
This solves three problems:
The first problem it solves is that it reduces the filesize of each download. They no longer have to pack in those shared textures and scripts, because they're a separate download. This means people have less they need to download for each character, and so they can download them more quickly.
The second problem it solves is the intended target, the overwrite-the-wrong-version problem. Because none of the characters include the shared files, there is nothing to overwrite. And because the shared files are an entirely separate download, maintained and updated separately, the user can download the shared files whenever they want, and automatically have them update every character that uses them.
The third problem it solves is one that was never even brought up before now, and is indeed a problem I never even considered, because it has been so deeply entrenched in Just Deal With Itβ’οΈ for me that I've internalized it: by keeping the shared files entirely independent, I can now update them freely. I have been so terrified of touching the shared files for years now, because of this conflict issue, I haven't even considered it 99% of the time. Any changes to them I have had to fret immensely over - are the changes worth the conflict issue? Now, I don't have to worry about it. If I think of a way to improve the shared files, I can just do it, and push a new release version. It won't break anything, and I won't have to worry about people losing track of the correct versions and managing their overwrites intelligently. It's all just handled by the content management system.
So that's dependencies. I hope you all can see the immense benefits it provides, both to you the end-users, and to me the creator. And I hope you can also see a little bit behind the curtain as to all the decisions and concerns I have to juggle when releasing assets in the first place, and why it's just so taxing for me. Why I am writing this system to manage it all for me in the first place.
I hope you can also see why it's been a bit of a clusterfuck for me to implement, especially in the user interface. I spent longer on this than I expected to. Certainly a lot longer than when I first imagined the dependency system as being "just a list of package names".
The Auto-Releaser - Descriptions
Beyond the dependencies system, the other major system I've worked on for the auto-release system is description generation.
A big part of the rigmarole that's discouraged me from releasing small off-hand models, focusing instead on big fanfare releases like full characters or large packs, is the requirement to sit down and write out descriptions.
Despite how it may seem given the Goliathan length of these posts, I don't actually like taking an hour of my day to sit down and write long elaborate posts for every little thing that 99% of people won't even read and then will DM me over Twitter asking things that are very clearly described in the text they refuse to read.
Writing descriptions for model releases is a chore, plain and simple. And so a compromise I've already accepted with the auto-release system is it won't support bespoke descriptions. In my mind, I imagine the interface for downloading auto-released assets to basically be that of Google Drive or OneDrive - a grid of zip-file tiles with names. No fancy sliding menus, no comprehensive animated gifs, not even a preview picture. Just a tile and a name.
But just because the descriptions won't be hand-written, doesn't mean descriptions as a whole can't exist. I'm not going to plug ChatGPT into the system (though there's a thought) to write fanciful flourishes of prose. But I do have access, by nature of the system, to every file contained within the package. And the system already has hard-coded preconceptions of what these files are - it knows, for examples, that models have related files like phonemes and materials. And it knows materials have associated files like textures and other materials they derive from.
It isn't much a stretch of imagination then to imagine a utilitarian description, a register of everything included, broken down by model, material, script, and texture. And I can go further than that, because I have a strong naming convention I adhere to. For example, models that have "dazv5" in their name are, by convention, character models. So I can label model with "dazv5" in their name as a character.
However, the system very quickly becomes more complicated then. What about hair? I name hairs after their character, after all. So Femshep's model is "Femshep_DazV5", and her hair is "Femshep_DazV5_Hair". By the previous system, her hair would be a character, too. But that isn't right. It's a hair.
From here, the rabbit-hole just goes deeper and deeper, and what started out as an imagined simple system quickly becomes a mess of spaghetti logic and exceptions and "always this except for when that unless also this".
So yeah. That ended up taking longer than expected. But hey - it works!
Mostly, anyways. It currently kind of breaks down with scripts. For example, the DazV5 IK rig has "dazv5" in its name, so it's considered a character. And the "Outfit Loader" has "outfit" in its name, so it's considered an outfit.
So that just means I have even more spaghetti logic to write - "ignore the 'character' and 'outfit' classifications for Python files". Such is life.
The Auto-Releaser - Prefabs
So now we're getting into territory I haven't broken ground yet. The system is great as it is. It's totally usable.
But you know what could make it even better?
Automating it even more!
As it is right now, to release a DazV5 model, I need to right-click the model and choose "Create new release". This starts a new pending release which I can add additional assets to. It automatically grabs all of the related model files required to get the model into Source, as well as its phonemes if they exist. And, of course, it automatically grabs its materials and textures.
But it doesn't grab the hair. So I have to right-click the hair and choose "Add to release". Even though the hair and the character model have similar names by convention, not all models follow that same convention - especially models not made by me. And conversely, not all models that just so happen to share the same name are actually related. So it would be uncouth of me to hard-code that association.
Not a big deal. Not super inconvenient. But now I have to grab the outfits. Those aren't grabbed by default either. See the above reasoning as to why. Luckily, the auto-releaser has the ability to add entire folders to releases. And conveniently, the way I do outfits is I place them all in a folder named after the character. So Femshep's N7 armor is in a filepath named "Femshep_DazV5/Outfit_N7".
So I right-click the "Femshep_DazV5" folder and choose "Add to release" to add every outfit in the folder to the release. As before, it automatically collects all the associated files, materials, and textures.
But now we go full-circle, back up to the top of this post: dependencies. As I started with, all of my DazV5 characters use shared textures and shared scripts. But of course, not all models use the same textures. A nice pair of leg textures doesn't do much for a water bottle. So I can't just hard-code every release to include the shared textures. Especially since the shared textures are a user-generated release. Again, that's just uncouth.
So now I have to right-click the pending character release, and choose Add Dependency. From here I am presented with a dropdown of every release made so far. Pretty straightforward. I choose "DazV5-5 Shared Files". The dependencies automagically resolve themselves, and those textures will no longer be included in the release. But I also need the utility scripts, like the Outfit Loader and Model Favoriter. Those are used by more than just DazV5-5 models, so they're in a separate release.
So I right-click the release again, and add another dependency. This time, I target the "Base Character Utilities" release, which contains the Outfit Loader, Model Favoriter, Material Overrider, and Material Bodygrouper (formerly named the DazV5 Bodygrouper, but it's long since been a general-purpose tool).
Strictly speaking, nothing changes in the release for this. None of those scripts were automatically grabbed by the tool. But it will prompt end-users that they need this dependency, which is valid. Just because the tool didn't strictly associate them, doesn't mean that they aren't practically used together. So I add the dependency.
Now all of a sudden, my one-click release has become a dozen or so clicks. And my automated system has become rather manual. It's still far more automated than before, and more automated yet than collecting everything by hand. But the goal of this auto-release system is to minimize the amount of work needed for me to release models, so I have no excuse not to release the small little assets I put together on a whim that people would like to have access to.
So the question arises: can I reduce those dozen clicks down to one?
The answer is "yes!". A new question: "how?"
This is how we finally get to this section's subject: Prefabs.
I'll be honest with you all. I don't know exactly how Prefabs will work on a technical level. I don't know how they'll look. How they'll interact with the systems. How users will create them.
But the idea of Prefabs is simple: they are collections of custom functions that are run on top of the existing system. Whenever you add a file to a release, the "on file release" function on your prefabs runs. Prefabs themselves are simple checkboxes you flip to turn them on or off.
The idea here is that I will have a "DazV5 Character" prefab. When enabled, it will check every file that's added to a release. If it is determined to be a DazV5 character by the spaghetti code described in the Descriptions check, then it will operate on a bunch of assumptions: it will include any "hair" model with the same character name. It will include every file in a folder named the same as the character (where by convention I put outfits). It will include the "DazV5 Shared Files" and the "Base Character Utilities", by exact name, as dependencies. It will do all of this automatically.
With this "DazV5 Character" prefab enabled, I can simply add "Femshep_DazV5" to a release. And it will automatically do all of the work described above, those dozen clicks, for me.
One-click release was the ideal. And with Prefabs, one-click release will be the reality.
I just have to, you know, get there first.
The Auto-Releaser - User-Side Content Management
But wait! There's more!
Everything I've described so far has been on my end of the auto-releaser system. But it takes two to tango. All of this means nothing if there's no way for people to actually download the files being managed by the auto-releaser.
This is where user-side content management comes in.
Thankfully, compared to the monstrosity of the creator-side content management, everything I've described so far, this is much simpler.
I haven't started on this yet, but the general idea is very simple: user-side content management will be split into three systems: Gallery, Browser, Manager.
The gallery is the top-most level of management on the user. It is what you all are used to. It will be a new page on my website, separate from the existing Assets page, and it will visually look very similar to Google Drive or OneDrive, as I described previously.
It will be a simple grid of zip-file tiles, with a name and the first line of the description beneath it, saying what kind of release it is (a character release, a prop pack, a script set, etc) and how many files it contains. When you click on a tile, it will expand to show the full description and a "download" button. You click download, and it downloads a zip file. Pretty straightforward, nothing revolutionary or groundbreaking there.
The second level of management is the browser. Now at first glance, this may seem equivalent to the gallery. After all, the gallery is viewed in the browser. And this is correct.
But the browser management layer is a special feature of the gallery. If you've ever used Nvidia's website to find drivers, it's basically that, albeit a bit more automated than the way Nvidia requires you to navigate a hundred drop-down menus.
At the top of the page, there will be a button called (name pending) "Find Updates". This button will then prompt you to locate your download metadata. This is something I have to figure out how I want to do it exactly - probably the most rigorous solution is to include it in the first nested directory of a download, so the models/ folder for characters as an example, to minimize the chance of users accidentally leaving it out of their installation.
Regardless of where I store the metadata, and how you as a user locate it, once the metadata is located, it will do a very simple operation: It will locate every release you have downloaded (and every dependency needed, regardless if if you've downloaded them), and locate them in the gallery page. And if the version you have is older than the version on the gallery, then it will create a new section of the gallery for you, at the very top, labeled as "available updates". And you can simply download them at your whim, and install them just like you would any other gallery download. All to make it easy to make sure you're up to date.
All of this would be done entirely client-side, meaning none of this information leaves your computer. I can't be assed to set up a server to process all of this, and frankly, it doesn't need to. Your computer has all of the gallery information anyways to display it, and it's a lot faster to just compute it all locally rather than send it off to "The Cloud" and process it remotely.
I have absolutely zero interest in data-mining your SFM installs. Frankly, I don't think I want to know what you sick ducks do with your SFMs. I barely want to know what I do with mine.
This naturally leads into the third layer of management, which is the dedicated manager program. This will function basically identically to the browser layer of user-side content management, with two major differences: it is a standalone program you can set up to run automatically on a schedule within Windows (it won't have explicit functionality for that, but it will include instructions as to how you can do that if you want), and it will manage the downloading and installing of updates (with your explicit permission) for you, without you needing to remember to visit the gallery.
More specifically, whenever the manager scans for updates, it will fetch the publicly-available data file for the gallery page off my website (basically downloading the web page), and then perform the same operations as the browser manager. You give it the location of your download metadata, it searches by release name, and it prompts you with available updates.
You check-mark the updates you want to download, and it will automatically download and install them for you.
The manager will also be able to basically mirror the gallery page, with a "scan releases" button. When pushed, it will present a list of all of the downloads available on the gallery page, from newest to oldest. Any files with a release date newer than the last time you ran the manager will be specially colored, so you can see all the new releases since the last time you looked.
The end-goal here is to take the savings I gain in uploading releases, and pass it on to all of you for downloading releases. It takes me one click to upload a release, it takes you one click to download a release. Everyone wins.
Primordium, Corsair, and DOAFACU
Oh yeah, I also worked on other things too. Not much to note here.
All of the dialogue audio for Primordium Episode 1 is finally in. Innie, as always, knocked it out of the park as Yennefer. Milly, Bordeaux, and Ivan similarly do absolutely fantastic performances as Triss, Ciri, and the Scholar respectively.
Innie is hoping to record all of the dub audio before the weekend as well. By all accounts, we should be able to finally install all of the audio this weekend. Very exciting.
And then I've gotten Medusa into Source. The poster image up at the very top shows them off together.
Medusa isn't short. She's actually perfectly average height (by design), being about 5'5 (165cm), which is the statistical average height for an adult woman in the United States. Artemis is just really fucking tall, standing at 6'3 (190cm) barefoot. This is, of course, because Artemis is the spiritual successor to Samus Aran, who also canonically stands 6'3 barefoot. Indeed, she is built on the same body as my DazV5 Samus, albeit with a few minor tweaks since that's such an old character model now.
Medusa's little tentacle things are fully prehensile. As a data operator, per described in a previous Progress Report, Medusa units are tightly integrated with computer systems. The tentacles are primarily used for distance interfacing, think of them as really long USB cables that she can control with her mind. She can also go full Doc Ock with them, and do silly things like spank Artemis with them. Artemis, being a shameless xenophile, gets quite turned on by Medusa's noodly appendages.
Finally, I've put together a few more outfits for the DOAFACU. Working on filling out the outfit matrix. Here is the current state of said matrix. Newest two outfits are the Leifang and Nyotengu Casual outfits, with a minor modification (new glasses) to Leifang's other outfits.
I have a few more outfits I plan to work on over the next few days as well, as part of the breaks I take from working on the Auto-Releaser System.
All in all, this system is going to probably take me a few more weeks to finish. Once it's done, it will be a huge Sword of Damocles lifted from my head, and I will jump into finally finishing Claire/Jill and getting it out the door. Feeling pretty excited to work on it again, but I want to get this damn system done first and get out from under the shadow of dozens of models I've promised to release but never did. It's been really eating away at me, even more than Claire/Jill has been.
I thank you all for your patience.
That's all for now. Honestly, I may not do a progress report next Monday. There's really not much to show with working on this tool. It's even more boring technical shit than I normally do.
We'll see how things go.
Until next time, everyone!
Comments
J Arco
So courteous of you to fill us in on all that you do. It helps us to understand how you spend your time and resources, and gives us insight into where our money is going and so we can appreciate all the work you have to do behind the scenes so we can enjoy (see: FAP) to your outstanding work. Cheers!
Raptor_Demolished
Thank for the update as always! Very happy to hear Innie got her audio in and loved seeing Artemis with her look and description. Very looking forward to what is to come π keep up the phenomenal work.
