Touhou-Project.com
patreon
Spoiler!
🕑 Added 2019-05-27 20:50:07 +0000 UTCIf you’ve been reading these posts regularly, then you’d know that the last time around I said I’d be working on the settings menu and on improving the browsing experience for mobile devices. That was the plan until something else annoyed me a lot. I was looking over an old archived story and was reminded of the fact that there’s no way of seeing images that have been marked as NSFW. All you get is that thumbnail as the large version of the image has been delinked and deleted. So I decided that this would be a good time as any to get that sorted. I knew that it wasn’t going to be that straightforward to do but experience in messing around with related systems made me confident I could find a good solution.
Let me first explain how the NSFW images (I’ll be calling them ‘spoilers’ in this post because that’s what they were meant to be code-wise) used to work. Whenever a post is made with a file attached to it, it gets processed, information about its dimensions is collected and a thumbnail is made (variable size if it’s OP or a reply post). The old system had a crude but efficient way of marking as a spoiler: if the spoiler checkbox was marked, thumbnailing would be skipped and instead a copy of the stock spoiler image would be copied over to where the thumbnail would normally go. That is to say, no actual new thumbnail was made for the source image. It’s relatively straightforward code and the main drawback is that the board software thereafter has no knowledge if the image is a spoiler and so has no way of it being manipulated after the fact. Not really a problem when you have a fast-moving chan-like board.
This code was copied into THP way back when people still developed and used Kusaba X and I added it in because, honestly, I had minimal knowledge of the board software’s workings at the time and I didn’t think I would be the one wholly responsible for the site years and years hence. It got the job done and that was that. My sole contribution was renaming “spoilers” to “NSFW”.
In order to “reveal” spoilers though, you need to have something to reveal. So I thought about the best approach for this. I like to plan things on pen and paper and you can see below how I approached the problem:
Behold my beautiful handwriting and occasional misspellings (Not pictured: another page of more notes on stuff to be done and a water stain)
Basically, two main things needed to happen: 1) Whenever a post is marked as a spoiler, it’s identified in the database and then that info is used whenever board pages are built, 2) Actual thumbnails needed to be made for existing, archived and future threads.
Neither was conceptually too difficult but the devil’s in the details with THP’s spaghetti code. First I got rid of the old code in the various files it was present. Then I modified the database to include a new column that tracks if an image is set to “spoiler”. After that it was a matter of add a case when the board is processing a new post to track if it’s a spoiler. And still after that modify the various templates that build the html pages to make use of this new data. So like set a thumbnail’s image to the nsfw stock image if $post.spoiler is true and otherwise build the post as normal.
Still with me? All that I’ve simplified here accounts for about half a dozen files and perhaps two or three dozen lines of code. Oh and as an aside: for good measure I added a pair of new columns to the board table in the database, to make it so the ‘spoilers’ and ‘update’ checkboxes are generated whenever they’re set to true (as opposed to it being an array with actual board names that got checked against at generation time).
Following on from that, it was time to lay the groundwork for the user-side scripts. The intended effect was if you click on a spoilered image, it would load the thumbnail and add a little disclaimer that it was a NSFW image. Then the plan was to generalize that and make it a setting that made it so every spoilered image would be automagically uncovered. This needed some alteration to other scripting functions (the image expansion code in particular) and testing. But several hours afterwards and I had a stable base.
The next thing that needed doing was point #2. It already worked for new images but what about old images? No two ways about it: I’d have to generate new thumbnails for every existing nsfw image.
But that’d have to wait as I’d need to manipulate the live site and I’d rather push all my changes through at once after thorough testing. I took care of many smaller things that needed doing (most of them are in the attached image under ‘bugs/misc’) which meant reworking old code and improving it. Naturally, testing was needed as well. While I was at it, I also continued my quest to remove outdated and useless code and simplify the mess that is KusabaX. Pretty good results on that front, incidentally. I ended up eliminating several dozen lines of code and a few files entirely without breaking anything else.
I also decided to tackle another outstanding issue that was sort of related to thumbnails: webms. This was also code that had been initially taken from somewhere else but, unlike spoilers, I had already made a few tweaks to it over the years. In a nutshell: spoilers had their own process for generating a preview image and they didn’t play nice with the rest of the board (thumbnails were not of uniform size, wouldn’t expand when clicked on and didn’t have information set in the database). I created a new function that added the bits that were needed for the database in future posts and altered board generation code to treat webms as a less special case and more like a regular media file with a couple of quirks.
In either case, I’d need to create new thumbnails that went along with all these changes for existing posts. It was the same for the NSFW images. Once I had done enough bug testing and finished adding bells and whistles (like the front page now reading whether or not an image in ‘recent images’ was marked as a spoiler) I pushed these changes live to the site. This was on Friday.
I wrote a special file that when executed would take from the database information on every webm file, run several functions and regenerate spoiler images (and delete the old ones). This went smoothly and took about 50 lines of code. As you can see, webms now play if clicked, can be expanded, close when the close button is pressed and also autoplay if you have the hover image expansion setting on.
The next step was to do something similar for live nsfw images. I created another special file, about 80 lines of code, that checked every thumbnail and matched it against md5sum of the stock spoiler image. If there was a match, it’d delete the old image, create a new (actual) thumbnail and then input that thumbnail’s dimensions into the database as well as the fact that it’s a spoilered image. The dimension information is useful when you’re revealing them so that they become their actual size instead of being smushed into the 100x100 that the stock nsfw image is set to. This process went more or less smoothly. But it did require some manual intervention as some files weren’t being processed. This was due to them being of a wrong filetype (like a .jpg that was actually a png) but I sorted the ten or so cases afterwards.
The final and trickiest bit was to deal with the images in archived threads. You see, with live threads any changes that are made on the database level can then be forced into new files. Archived threads either no longer exist in the database or have misleading information (ie: posts are marked as deleted and wouldn’t show up if you try to regenerate it). Furthermore, the source images from which the thumbnails are generated are deleted as soon as they’re archived. So another approach was needed.
I divided up the work. I wrote another similar file to the previous ones that read all archived thumbnails, checked their md5sum and if they were the stock spoiler then their path was exported to a file. As I have a (mostly) complete backup of THP locally, I then whipped up a script that copied all those (deleted on THP) images to a directory on my computer. I uploaded the 300ish MB result back to THP. While that was going on, I wrote up a script that read the file list and modified the html files and added the new information I needed for the thumbnail revealing to work. For the sake of completeness: the image file got a new class, its source was set to the stock spoiler image and the ‘true’ thumbnail was added to as extra data which is read by the user script when you click on it and replaces the stock thumbnail.
There were a few images I didn’t have a copy of or no longer were in an archived thread (some get deleted because they’re not stories or whatever else). But it’s ~30 out of several hundred.
Finally, another special program was run that created thumbnails from the images I uploaded and copied them to the right place. That last bit was done earlier today actually. I’m very satisfied with how things turned out!
Though this was a conceptually simple change, it still represented many hours and days of work but it was worth it. In the future manipulating spoilers or webms should be easier. The code base has been streamlined in many places and many features improved. A sizable number of bugs have also been quashed. All in all, there are about 1600ish spoiler images that were affected by this, around 70 webms. 10-15 board software files and hundreds of threads.
It’s still my intention to improve the experience for mobile users next and, to tell you the truth, I’m still debating how to best approach that. With solid groundwork in place, however, it will probably be easier than it would have been a year ago. I may also take the time to refactor and simplify even more code that I danced around when making these changes but that’s a secondary objective.
Until next time, take it easy!
Comments
Benjamin Oist
All that work and attention to user experience for a bunch of us lunks who very occasionally write/read/vote on frilly hat girl stories. We don't deserve you, Teru-kyun.
