Overlooked way to decrease oblivion loading times

[angevil]

I just bought a older model SSD, but new (not used), and which is fairly slow by today's standards. In average i noticed an increase in speed by 3x, like bootup times or game loading times. However i saw that Oblivion had only around a 30% increase in loading times, and i decided to find a way to decrease them, since this is the primary game i play atm.

I have cpu usage displaying at all times, including single threads, and i noticed that oblivion when loading saves or when entering and exiting buildings was not limited by HDD (it was more when i had the game on the mechanical drive), but by the cpu, which was running one core at 100% during loading. This meant that the CPU was busy decompressing all the BSAs with meshes and textures.

Using obmm, then i extracted all the meshes and texture from all the BSAs, including mods, like Better Cities and All Natural. Then i copied and pasted all the extracted files in the oblivion folder, without overwriting, since that meant that i already had the needed meshes or textures, which came from mods that further improved what was in the BSA in some ways, like a HD texture pack. After that i moved out the BSAs out of the oblivion folder, which i will keep for now somewhere else for backup purposes.

After this, loading times decreased by 50% compared to what i already had. This applies to all loads, save and quicksave load, and entering and exiting buildings. For example if something loaded with the mechanical drive in 10 seconds, then before I extracted the BSAs it loaded in 7.7 seconds with the SSD, and then after i extracted the BSAs it loaded in 5 seconds. Also i noticed an improvement in fps of around 5% in most exterior areas.

[vashar]

Increase in loading times..?

I think you mean decrease. But unpacking the BSA files will improve loading times, and in FPS? Interesting.

[gerra6]

Increase in loading times..?

I think you mean decrease. But unpacking the BSA files will improve loading times' date=' and in FPS? Interesting.

[/quote']

It makes sense. If the BSA uses any compression at all (note that this shouldn't apply to uncompressed BSA archives), then the resources will have to be unpacked in order to be used. This would certainly increase loading times. The amount of a performance hit would probably depend on the degree of compression.

There also should be a similar framerate hit whenever a resource needs to be loaded into memory from a bsa.

[cobme]

Very interesting , I'll try that myself.

Many thanks!

[angevil]

Very interesting ' date=' I'll try that myself.

Many thanks!

[/quote']

You're welcome. I've also PyFFIed the whole meshes directory which included all the meshes from the extracted BSAs (4.2 gb, 27k files), and it was no improvement... i noticed a 2% decrease in fps in a benchmark spot, but a 2% increase in fps in another benchmark spot.

Other than the faster loading times, i don't notice stutter anymore outside when running around, because the cpu does not work anymore to decompress the BSAs. The only thing is that the memory usage is higher than before. This is not a typical scenario, since i have every HD texture mod i could find, including LOD. 8GB of ram would be more than enough, and this would never be an issue then, but i have 4. Because of this I had some stuttering from the pagefile being accessed, because the ram usage went over 4GB, so i spread the pagefile over 3 HDDs (not drive letters), and that fixed it.

[Sarnaath]

(note that this shouldn't apply to uncompressed BSA archives)

I was wondering if just repacking the BSAs without compression would accomplish the same thing. But I went ahead and extracted and installed all the BSAs I had (18). Took quite a bit of time since the Oblivion ones are so huge. I'm not really seeing a noticeable FPS increase. Loading times, I need to play more. I didn't rigorously test that. I think I'm going to repack all the file back to BSAs, uncompressed. Wrye Bash seems to take longer to open the installer tab now, with so much to scan.

[gregathit]

Uncompressed bsa files should indeed give a bit of a speed boost.

[Sarnaath]

Well, it turns out that all my BSAs were already uncompressed. A month or so ago I had extracted them all, ran the meshes through PyFFI, the textures through the Texture Optimizer, and recreated the BSAs. Apparently I did so with out compression. The sounds & voices BSAs had no file size change, so either compression doesn't do anything for them or they were never compressed to begin with.

That probably explains why I didn't see a loading time or FPS change. Nice tip though.

[gregathit]

Most of the sound and voice files should be mp3 (which needs no further compression) or tiny wav files that probably would not benefit anyway.

You should have notice a 1-10 FPS bonus with PyFFI on your meshes and textures however....unless you are running a bunch of massive textures that is...

[Sarnaath]

Most of the sound and voice files should be mp3 (which needs no further compression) or tiny wav files that probably would not benefit anyway.

That's what I would expect' date=' if I had thought about it ahead of time. But I really didn't know what all was in those BSAs.

You should have notice a 1-10 FPS bonus with PyFFI on your meshes and textures however....unless you are running a bunch of massive textures that is...

I did see a bit of an improvement when I PyFFI'd the meshes. Some of it may have come from replacing the compressed BSAs with uncompressed BSAs too. Once I PyFFI'd the meshes though, I reinstalled Better Cities without the FPS patches, which put me back close to what I was before, at least in Imperial City shipyard area.

edit: yeah, running Quarls Redimized. No FPS difference through without it though.

[gregathit]

Yea, the waterfront is pretty much a FPS kill zone regardless of what you do. With mods that add to it like Better Cities it can slow you to a crawl unless you have optimized your settings and\or you have a pretty decent computer.

It took me quite a while to fine tune my settings to have a decent frame rate at the waterfront, especially since I installed Qarl's texture pack.

[vashar]

I think I should try this decompression thing :B

[gregathit]

I think I should try this decompression thing :B

It certainly won't hurt you.

[Slammer64]

I know this thread is a couple of months old, but for speeding up Oblivion/FalloutNV/Skyrim you CANNOT beat an SSD, just got one and installed it, I cannot believe how much it speeded stuff up. I'm going to spend the rest of the night enjoying Oblivion like I've never seen it before. 

[Symon]

Be aware of the limitations of an SSD. Check the number of writes they are certified for.

RAID 1+0 (aka RAID10) with four HDDs is a pretty good alternative. Not as fast, but more robust.

 

To mention the BSA issue, Beth use them for easy monolithic installation, not performance.

 

Consider:

If your textures (say) are loose, when the game needs to load one into memory, it makes a call to the operating system to retrieve it. The NTFS file system does a swift job of that and the texture (say) is loaded into memory with a few o/s calls and processor ops.

 

If Your texture is in a BSA, the game makes a call to NTFS to find the BSA, then it scans along it (likely more calls to NTFS) (maybe decompressing as it goes) extracts the texture and then loads it into memory. The same calls to the o/s and processor ops AND more to deal with the BSA. A BSA is not a magic free lunch! It requires just as much effort to read off disk as loose and more!

 

That's a lot more Disk operations and processor operations, especially if compressed. Think how long it takes to use windows explorer to find a texture and then open it in photoshop as opposed to say BSA commander to extract one and then do the same.

[Bashis]

word to the wise, dont right click your data folder and tell it to ppyfi the whole thing unless you have ALOT of free time, and before you tell my that i shouldnt have done that, i know its not the smartest thing in the world to do the whole 70 gigs in one shot, but i have backups

[Symon]

Also, the last time I read the PYFFI, readme, some mesh types should NOT be optimized with it. (Hair for one IIRC).