1. The space forums use a separate registration system from the main virtual world. To comment on these forums, you need to create a separate forum account.
    Dismiss Notice
  2. Hey there, if you want some direct help without hopping inworld - try our Discord channel at https://discord.gg/mptfycQ
    Dismiss Notice

Bug Report Furniture Item File Size increases inworld

Discussion in 'Bug Reports' started by ghaelen, Dec 28, 2017.

Share This Page

  1. ghaelen

    By:ghaelenDec 28, 2017
    Moderator
    Staff Member Moderator

    Joined:
    Sep 29, 2016
    Messages:
    65
    Likes Received:
    20
    I uploaded a decorated, animated Christmas Tree with a file size of 8.39mb.
    In the MP inworld, though, it says the tree is 19.54mb.

    This seems backwards from the wonderful crunching Space does on the regions. Is there anything I can do to prevent this? I hadn't noticed it with other items (of course now I'll go back and check them :) )

    treefilesize.jpg treefilesizeinshop.jpg
     
  2. Adam

    By:AdamDec 28, 2017
    Developer
    Staff Member Moderator

    Joined:
    Aug 21, 2014
    Messages:
    702
    Likes Received:
    139
    Hey, the processed and final sizes can be different - it's unusual for something to be larger than the upload size; but you can use the filesize inspector tools (use a blank region) to figure out why something is the size it is.
     
  3. Trilo

    By:TriloDec 28, 2017
    Active Member

    Joined:
    Dec 6, 2016
    Messages:
    318
    Likes Received:
    46
    The tree file size you're seeing may just be the model itself, and not include the textures? Textures are where I normally see the biggest increase, especially with PBR materials (since it's the size of the texture map times the number of maps being used in the material). Crunching can help (over uncompressed), but it's still the biggest factor.
     
  4. ghaelen

    By:ghaelenJan 8, 2018
    Moderator
    Staff Member Moderator

    Joined:
    Sep 29, 2016
    Messages:
    65
    Likes Received:
    20
    I did some extensive testing, and I can confirm that furniture items don't go through a crunch process like regions do. Their upload-to-inworld filesizes differ from item to item. The tree above is heavier inworld than as an upload, but my huge dance globe with 5 animations and a 5400x3700 NASA texture weighs the same as it does during upload, in the region, and as a virtual good - 3 mb. (Shop item here if you want to verify that) The animations together are about 3mb and the texture jpg is 3mb on its own.

    Test Candle:
    1. One fbx mesh, 3 materials, 6 - 512 textures spread across all materials.
    2. At least three of the textures are repeated 3 times. One of the normal maps is repeated twice (albedo and detail) on each of two of the materials, for four instances of the same texture.
    3. Child object (particle fire) with a texture sheet (1024x512) and 2 particle textures (256x256 and 64x64). I modified the fire in pyroparticles from the unity asset store for anyone who knows how heavy that system is.
    4. Add 3 store images for the candle in the virtual good script (430, 96, 120,etc)
    • Uploaded as a virtual good item, the candle uploaded at 2.7mb and weighs 9.85mb when inworld. (Shop Item Hereif you want to verify)
    • Added to a region and uploaded it adds less than 3 mb (total region ~5mb).
    So something actually increased when it was extracted from the zip as a virtual good/furniture item.

    Maybe the textures in Virtual Goods don't repeat so each texture weight is added, rather than nulled out if it's repeated.
    Edit: I think this pertains to furniture items, because regions and gestures seem to get crunched.

    • Finally, I uploaded the same candle, same number of textures, using two substances, one which was copied to make two materials. The VG item size increased from 9mb to 24mb. The region size increased from 7mb to 17mb.
    (I'm sure I did the substance wrong, or shouldn't substitute my own textures for the native ones. I understand they are supposed to save on weight (Trilo, I watched your video here on the forum) For now I'll go back to my simple 512 textures. They actually looked as good.)

    **I LOVE the website storefront, btw. Waiting til customers can buy from the website and collect their items inworld***

    Having worked my way through this, I will add two things to the feature request in the new Mantis site: :)
    1. Crunch virtual good items if possible? Or tell us what to do so they don't balloon in size when inworld
    2. Add the inworld weight of a region/virtualgood to the data that shows on the website curator. I couldn't make heads or tails of the 'upload-to-inworld size' result by looking at the editor info. I had to put a price on the object for it to appear in the shop, because that's the only place virtual good file sizes appear.

    This will all be very important, not only for large items, but for small things like this candle that will affect the filesize of a person's inworld space. Customers who understand their limits probably won't buy a heavy small item. But as the creator, I don't know why it becomes heavy. I made it as low poly as possible and assumed it would be crunched like regions and come out even smaller than the 3mb upload. To see it tripled in size was (and is) concerning.

    Fortunately, setting out six of the same candle did NOT increase the region size sixfold. So 'something' repeats when inworld. Huzzah for that at least!
     
    Last edited: Jan 8, 2018
  5. Trilo

    By:TriloJan 8, 2018
    Active Member

    Joined:
    Dec 6, 2016
    Messages:
    318
    Likes Received:
    46
    You should be able to inspect the costs on a per item and texture basis by putting it into a test region, then in the Scene Export Settings click on the Filesize Inspector (Output). But if you check in your regions, you may notice the standard filesizes per texture resolution. Based on full color maps, I see these standards...

    2048x2048 - 5MB
    1024x1024 - 1.33MB
    512MBx512 341K
    256x256 - 85K

    In cases of grayscale maps, the filesize is about 1/4 of that. Those are generally fairly standard numbers. In SL it came in a little less because they didn't do mipmaps (they're generally a really great thing, more info on that can be found in this wiki article)

    Multiple instances of the same texture being used should not mean it takes up mulitple times that filespace, since it's all just referring to the same file.

    As a user and content creator, NO PLEASE DO NOT CRUNCH CONTENT ON VIRTUAL GOODS! Maybe make it available as a very optional choice for creators, but even that idea makes me wince (aside from lossiness, the time it takes to decompress means increased loading lag for customers as well as anyone in the region that a user teleports into.

    I'm not sure how using .sbsar files would affect a virtual goods item (they're different creatures than regions/scenes), or if they'd need to be baked to the item during processing (perhaps Adam can clarify). Even on regions, using a substance can depend on the filesize of the substance. Most of them are very small files, and for the cost of that little file it can bake multiple maps (albedo, normal, height, etc), but I've seen some that are pretty massive. How big were the .sbsar files the item was using?

    I haven't yet made any furniture items, so I don't have a lot of experience or tips there to share. Generally for anything that's wearable or spawnable (rezzable, to use SL lingo) you want it to be as ready-to-go as possible, because there's no telling when an avatar is going to jump into a crowded room wearing it (or decide to spawn it in a crowded region).

    I did notice, and have raised an issue with the filesizes of the marketing pics (store and inventory and brand pics) is included in the item filesize calculations. In my testing an untextured white cube came in at over 1MB by simply using the new virtual goods snapshot tool. I talked to Adam about that, and he said that they would be changing it so that the store and inventory pics would not count against the item's file size. That should be helpful for region builders, furniture/landscaping builders, and users alike (1MB doesn't sound like much, but that times the number of items in a user room means you run out of space much more quickly than you'd like on the free and first subscription plan).
     
  6. ghaelen

    By:ghaelenJan 8, 2018
    Moderator
    Staff Member Moderator

    Joined:
    Sep 29, 2016
    Messages:
    65
    Likes Received:
    20
    Yes, I added up all the numbers for the textures I was using. They add up to less than 2mb. The size inworld is 9.85. I see a lot of MAT and GO but nothing else that adds 7mb of weight. 884 verts.

    You mentioned the mesh, so I looked at that but I don't know what a 'vert' weighs once inworld.

    The upload file size pulls in everything for the region, and when I add up everything pertaining to the candle, it adds up to about 3.5mb

    EDIT: I figured out the reason the substances made the file so big - they were big in the upload inspector as well. I am trying a different substance to see how it affects the inworld filesize.

    EDIT2: Using the same substance for two of the three materials cut the inworld size down to 7.95mb. In the region it changed the region size by about 2.5mb.

    That's the point I'm making. If you are right, something else is the culprit. But I don't know what. Right now it looks like it adds each instance of each texture.

    I upload furniture, regions, animations, bundles (but not clothes, lol). Only the furniture increases in size. I didn't think clothing added to the region size. I have pulled out inventory onto my floor in the middle of a crowd, so that might happen with both clothing and furniture. But this issue really seems to affect furniture that people would place on a room floor.

    I don't think our avatars have a file size limit for what we're wearing. Do we?

    So I'm not sure what crunching will mean - except to
    1. avoid INCREASING the file larger than it is in the editor. That is really the issue.
    2. repeat textures used anywhere in the same furniture object. Most of my furniture items are in an empty and there might be several objects in there. If one texture used in four child objects is treated as four textures you can see the problem.
    3. Let me see how big it is. I DO have to care with furniture items because the room size is where we are limited.
     
    Last edited: Jan 8, 2018
  7. Trilo

    By:TriloJan 9, 2018
    Active Member

    Joined:
    Dec 6, 2016
    Messages:
    318
    Likes Received:
    46
    I know they don't multiply on regions or on the virtual goods test items I've done, so unless something has broken in the last couple months I don't see it as being the culprit.

    Unlike textures, I believe that mesh/model size is based somewhat on the filesize on your local hard drive. More complicated models tend to make for larger files.

    Additionally, your item may or may not have shaders to factor in. If your item uses one of the SineWave shaders that Adam has included in the editor pack, then the item won't have the cost of additional shaders. Different shaders are probably different sizes, but on the handful of clothing items where I've tested the difference between SineWave built-in versus some other shader, it's about 4-5MB per shader.

    Crunching means to apply a lossy compression at the server level. Think of it as like jpg, except you have less control over the results. Compressed textures also take slightly more time to load than uncompressed, since the user's client would need to unpack each texture when loading. Actually, all users in the region/scene where the worn or spawned item appears would then have to take that hit on their end. That one's a real 'your mileage may vary' issue, it depends on the user's hardware (and probably hurts a lot for web client users).
     
  8. ghaelen

    By:ghaelenJan 9, 2018
    Moderator
    Staff Member Moderator

    Joined:
    Sep 29, 2016
    Messages:
    65
    Likes Received:
    20
    Thanks, Trilo, that is all very helpful and gives me other variables to check and to consider while creating the item.

    I suppose, too, that the upload size as 'zipped' will naturally be smaller than the extracted size. And I'm going to assume that the texture instances are not accumulated (e.g. 341KBx3 does not equal 1023). So I can (for the moment) assume the weight, or what you call 'cost' of the textures equals the sum total of one use of each texture for each furniture item nested in the same parent object/empty.

    That's interesting about the shaders. I try to use the Space shaders first, and I've used text sweeping for non-text images and a skin shader for the earth globe. I'll try one of them for the candle and see if it makes a difference.

    How does the choice of shader affect the use of substances? I've read that substances mainly use the standard shader and all those I'm studying from Substance Share do as well.

    I don't see a Space standard shader - is there one, or one like it? I'm probably confusing this issue as well but if not then I'm getting mixed messages - substances really reduce file size, substances use a non-Space standard shader (which adds weight back).

    Ah, then I agree that 'crunching' isn't the term I want to use.

    I think after all of this the one thing that I'd still like is to see the size info of the item when inworld on the curator or in our inventory. That's because the limit we're under is file size (rather than # of prims). So file size is what I am conscientious about.
     
  9. Allen

    By:AllenJan 10, 2018
    Administrator
    Staff Member Moderator

    Joined:
    Aug 22, 2014
    Messages:
    41
    Likes Received:
    5
    Yeah, except for texture and mesh, the shader might add some file size, the more shaders you used, it might add more file size, even same unity standard shader, different mode might add different file size. We ever tried to include the unity standard shader to the main client, but it added several hundreds MB to the client because it included all variants of the standard shader.

    Another trick, the filesize is the max filesize for all platforms, unfortunately, by default, the webgl version might have biggest file size (we optimized it for clothing, auto crunched the texture size for webgl etc, but we cannot do it for other virtual good because each virtual good might need different texture resolution). But you can do it manually by crunching the texture size for webgl etc in texture import settings. (Do not do this for the images (icons, preview images etc) used in virtual good compoment because they will be shared between different platforms).
     
  10. Adam

    By:AdamJan 19, 2018
    Developer
    Staff Member Moderator

    Joined:
    Aug 21, 2014
    Messages:
    702
    Likes Received:
    139
    I just want to put some cliff notes in here - this is a rather useful thread, which I think I needed to respond to earlier (sorry!):

    When a asset is uploaded generally speaking the upload size should not be larger than the processed size, however there are cases where this can happen, they typically are:

    * Shaders with lots of complex feature variants - these can easily consume vast amounts of space - a shader with 8 options will have a minimum of 256 times the normal variant count (which accounts for lighting/fog/etc.) - often turning something with a few hundred small variants, into tends of thousands of small variants (which suddenly add up).

    * Items that dont compress well in DXT - DXT is a bitmapish format - that is, it's typically around 5 bits per pixel - regardless of what is stored within them. PNG and JPEG are more content dependent, so content can compress better if it is substantially similar - so a 4096px texture which is say pure green, will end up a few kilobytes as a JPEG, but may end up 30mb+ in DXT. (That is a rather extreme example). Texture crunching will help here - it implements a simple filter over the top which brings the sizes back down to something reasonable - but as Allen notes, WebGL doesn't support that right now, so that will hurt there. I will however note, we should be ignoring WebGL size when working out what we treat the item as, as the crunching issue is extreme.

    * Items which unpack to uncompress well - as a variant of the above, animations and meshes both also have certain corner cases where compression does not work particularly well.

    I am in the process of testing a new shader pack at the moment which will shrink sizes substantially for the first category of problems - versus the Standard shader, you can expect a 1-3mb reduction per item.