Parsing, Components, and Workflow

[dalomar_cg]

Since I'm not a native SketchUp user, I figured I'd ask all of you guys for your opinions on some things that have come up within some of our discussions in the office.

As all of you know, our scene parsing is one of the biggest issues that we'd like to get going in a much better direction for the next full version of VfSU. Hopefully, you are all aware that components are one of the main reasons for this slow down, partially because they are slower to parse and partially because they are so widely used. Components could possibly be a time saver for us, however there are some issues with SketchUp's implementation of them that are causing the problems for us. As it stands right now, components are a a way to instance geometry, but the materials (and UVs) of components are unique. This means that we cannot treat components as instances because as far as we are concerned, we cannot be 100% sure that the geometry/material pairing are the same across all of the instances.

This brings be to my question, and the point of the discussion. If we treated components as true instances (retrieving both the geometry and material information from the base component), then that could possibly increase the speed of scene parsing. However, this would require a change in workflow, where a user (you guys ;D) would have to make sure that any material adjustments and changes were made to the base components themselves and not applied on top of a component. Is this something that you guys could possibly work with or would that cause too much of a hiccup in the modeling process? I want to get all of your feelings on this before we invest any time on something that you guys might be absolutely against.

[thomthom]

Re: Parsing, Components, and Workflow

hmm...
The materials doesn't seem to be much of an issue. As currently the workflow is already like that to avoid faceted surfaces that should be smooth.(plus shorting down the parsing time)

But, component instances are often scaled differently. And this would cause major headache if the instance scaling was ignored. I often make a component out of a profile which I then stretch to various length. That way I can modify the profile once and have it cascade through the model. I do this allot. And I see it down often in other people's models. And I know that when projects gets passed from person to person it's no way to know how it's done.

Getting the scaling/stretch/skew correct for each instance is a must. If I would have to create a unique component for each scaled instance my workflow would be completely broken.

[dalomar_cg]

Re: Parsing, Components, and Workflow

I believe that local transforms of a component should be able to be retrieved (position, rotation, scale). Skewing I am not 100% sure about, but if it can be reduced to a transformation matrix, then it should be able to be applied to the component.

[thomthom]

Re: Parsing, Components, and Workflow

Scale/Stretch/Skew are all stored in a .transformation class for each instance, so that sounds positive.

So are we basicly talking about only materials which are applied to the group/component instance which is being lost then? Or is there other properties?

[Jackson_cg]

Re: Parsing, Components, and Workflow

Damien,

Before I comment, can I establish one thing- when you say "components" do you mean just "components" or do you mean "groups and/or components"? Thom and I have been discussing this very issue over at SCF and it'd be great to clear this one up.

If I would have to create a unique component for each scaled instance my workflow would be completely broken.

Regardless of my first question, I second Thom's statement.

[dalomar_cg]

Re: Parsing, Components, and Workflow

I asked Joe, and he said that groups and components would be considered essentially the same within this context. The transformation applied to a component would not be an issue at all, as that apparently it already part of retrieving the final location of component verts.

[thomthom]

Re: Parsing, Components, and Workflow

Well that's good news. If it is only the group/component material that's the issue here, then I'll give my thumbs up. (Though it'd off course be nice to have that feature it can be sacrificed. )

[Jackson_cg]

Re: Parsing, Components, and Workflow

I asked Joe, and he said that groups and components would be considered essentially the same within this context.

I'm still confused- I know you know all the following, but I'll explain my understanding of Gs & Cs so you know exactly where I'm coming from. Groups and components are used very differently in SU- all geometry in a well-contructed skp file is organised within groups, but only a few of these will be "promoted" to become components, usually as they represent identical repetitive elements in a model, e.g. trees, handrail uprights, etc. Groups are dumb, they don't really perform any role in SU modelling other than stopping bits of geometry sticking to other bits of geometry. They do of course have other advantages, easier to select, easier to rotate, scaling UV mapping, etc, but they're essentially just "lumps" of geometry. Vitally, components allow simultanous editing of all intances so every balcony, or handrail on a balcony on a building can be edited at any point and the overall result viewed instantly. As far as my non-coding brain understands it, the only way that a component can differ from other instances of that component is by scale, both uniform and non-uniform: a modification that necessarily applies to material UV scaling too. An example of this which I use often is trees which are rotated and non-uniformly scaled, but all of which can still be simultaneously edited from within any one of their instances.

So my thoughts are:

Groups are by definition unique, with unique geometric contents and unique texture mapping, even if they've been copied. VfSU must recognise that ALL groups and their textures are unique. To treat them like components and start applying or reading/parsing them as though some were instances of the same geometry would be a major conflict with SU modelling practice. Put simply, every SU/VfSU model I've worked on would be wrecked.

Components are by definition identical instances of the same geometry with the same texture mapping and can only differ in overall scale. VfSU can pretty much do what it wants with them- I know exactly where all my components are in every model I work on and why they are components. As long as VfSU can recognise the modified scales (and of course the corresponding modified UV scale) of individual intances of otherwise identical components then it'd be no problem regarding standard SU workflow to treat them as "true instances". If this can speed up parse times... GREAT! ;D

[thomthom]

Re: Parsing, Components, and Workflow

Exposed to the ruby API, groups and components are the same. Groups have a definition, stored in the same collection which components does, even if there's only one copy of the group. When you copy a group multiple times and do not edit any one of them they are all instances from the same definition. This is how SU's inner working treats them at least.

So the question is Jackson, do you expect groups where the geometry has the default material to inherit the group material?
If so, I cast the ball back to Damien; Since groups and components are nearly identical in the ruby API, would ensuring that group geometry inherited the group material also slow down the parsing. (I'd be very surprised if it didn't.)

[Jackson_cg]

Re: Parsing, Components, and Workflow

Exposed to the ruby API, groups and components are the same. Groups have a definition, stored in the same collection which components does, even if there's only one copy of the group. When you copy a group multiple times and do not edit any one of them they are all instances from the same definition. This is how SU's inner working treats them at least.

I found that out from you recently, but it doesn't change the fact that until a component is made unique, it will always be a simultanously editable instance in SU and VfSU has to recognise that to be useful as a plugin. Whether the Ruby API treats copied unedited groups as instances of a component doesn't seem particularly relevant, firstly as, as soon as you make the tiniest modification to a copied group (and presumably even if you subsequently manually undid the modification) that group would become "Ruby API unique" and secondly, because it is very, very rare that I have identical copied groups in a skp file which I wouldn't have made into components instead. When I do, you can probably hear me swearing and kicking myself from Trondheim!

So the question is Jackson, do you expect groups where the geometry has the default material to inherit the group material?

I didn't understand that to be the question at all. :-\ The answer to that question would be "yes", in just the same way that SU does.

[dalomar_cg]

Re: Parsing, Components, and Workflow

Okay, I'm glad I asked...

Groups and components are used very differently in SU- all geometry in a well-contructed skp file is organised within groups, but only a few of these will be "promoted" to become components, usually as they represent identical repetitive elements in a model

I actually didn't know that there was that much of a distinction between groups and components. I actually thought components were much more widely used than groups.

@ Thomas
So when unedited, ruby sees groups as being instanced?

To clarify a little bit about my post above, Joe was referencing the the method in which we "open up" components and groups, and how we find all of the information within them. That is essentially the same operation between the two as they currently require applying the transformation that SU has within the instance definition.

At this point I don't see any reason why groups would have any issues with this change. Either A) they're all unique, in which case there would be no issue with instancing cause they wouldn't be or B) only the unmodified groups would be instanced, which again wouldn't matter because they would be the same as the instance definition anyway.

The real issue comes with components. As long as no additional material assignments are done to components on top of component itself (ie the appearance of the component is the same as when it was created), then there won't be any issues and we can treat components as true instances. However, this would mean that either A) material assignments that are made on top of the component would not be recognized and/or B) components that had these addition material assignments would have to be made unique and "redefined" with those material changes in order to register correctly or C) that particular component would just have to be exploded.

My question to you guys is are the last three points I made feasible in terms of workflow. Would it kill a SU user to have to refrain from applying materials on top of a component? Would an average SU/VfSU user be able to figure out that they can't perform that additional material assignment?

From the responses from both you guys, it seams like this is a sacrifice that you could make for better parsing. I'm also getting a hint that a "clean" model wouldn't have any of these additional material assignments on top of components anyway.

[thomthom]

Re: Parsing, Components, and Workflow

I actually didn't know that there was that much of a distinction between groups and components. I actually thought components were much more widely used than groups.

It depends on the user. I see people using groups all over the place because they aren't really aware of the difference and benefits with components. I've also seen people that only make components because by default the hotkey to create a component is mapped to G and groups doesn't have any.
Personally, I use groups when I don't have any repeating elements. The reason for this is that it makes it easier to navigate the Component Browser. There's no need to have a large number of components in the component list for the model which you will never use. I some times use components for single instances if I want to export/import various variation of a part of the model.

@ Thomas
So when unedited, ruby sees groups as being instanced?

Group copies, yes. The moment you edit one it's automatically made unique and a new definition appear in the definition list.

From the responses from both you guys, it seams like this is a sacrifice that you could make for better parsing. I'm also getting a hint that a "clean" model wouldn't have any of these additional material assignments on top of components anyway.

Not really. Applying colour to the group/component is actually quite a neat feature in SU. For instance, populating a parking lot with cars. You only need a couple of different type of cars, but painted with many different colours without having a definition per colour. I'd say it's cleaner to have less unique instances.
But, IMO it's a minor inconvenience and not a show stopper. As I mentioned earlier, I already try to avoid that due to issues with the current version.
You would benefit from explaining this in a very easily available FAQ. On this site or in the manual. A small list of the most common gotcha's. As I'm sure there will be users that will be confused about this. You can see this in the way backfaces works.

On a slight sidenote; I've been thinking about making SU plugin which would iterate over the model and try to spot some common potential issues. Like group/material colours, deep nesting faces with transparent material on one side but not the other etc. Just something that's return a quick report.

[Jackson_cg]

Re: Parsing, Components, and Workflow

Thom is absolutely spot-on with the above post.

The car example is particularly to-the-point as it's one of the occasions where SU's heirarchical "paint default geometry by group/comp material" system is extremely effective.

As Thom says, VfSU painting all identical components with one material is a problem in relation to SU workflow. However, as myself and Thom (and hopefully many others) use Matthieu Noblet's "Remove C-G" ruby script in our daily workflow I've become accustomed to applying materials directly to geometry and making some components unique to allow VfSU to recognise their different materials so it's less of an issue to those who are "in the know". For VfSU newbies though it would probably result in countless "What Happened To My Materials?!!!" threads.

It's actually my understanding that Matthieu is working on adding a function to his "Remove C-G" script which will automatically make identical components with different materials into unique components (thanks to Thom's excellent suggestion! ;D). In fact, it might be useful to have it as two separate scripts as not everyone is willing to "screw up" SU's default material system for the sake of faster parse times, but most could probably live with differently coloured comps becoming unique when prompted. If every VfSU user ran this script before beginning to render in VfSU, then I guess ASGvis issues with comp materials would effectively be solved, or at least VfSU could safely treat all identical SU comps as true instances.

Regarding that, I do have one further question- as the only attributes other than different materials which can differ amongst identical comps are uniform and non-uniform scale (and corresponding scale of the UV mapping), if VfSU treated them as true instances would it still represent scale and UV scale differences? It would be a serious issue if it didn't as many otherwise identical comps are scaled differently- trees, background building window frames, etc and to suddenly render these at their default scale would cause havok.

As Thom says, this and many SU/VfSU issues would certainly benefit from being explained in a simple FAQ, or "Getting Started in VfsU" pdf. Seamless plugins are a nice idea, but I haven't found one yet which didn't require at least some changes to SU workflow so it's not like VfSU would be putting itself at a disadvantage. As long as it's explained quite clearly that to get the best out of VfSU you should follow some basic "best practice" rules in SU nobody should have anything to gripe about (and we might see a lot less "Groups Disappearing??!!" and "Why Is VfSU So Slooooooow???!!!" topics every week ).

Hope this helps,
Jackson



p.s. any chance of making Matthieu Noblet's "Remove C-G" ruby script thread sticky? It's 10 pages deep now and yet VfSU SR1.5 is almost unusable without it IMO.

http://asgvis.com/index.php?option=c...29118#msg29118

[dalomar_cg]

Re: Parsing, Components, and Workflow

Okay, I've been a way from this for a few days and am trying to wrap this up for myself.

@Thomas
I popped up my Ruby console to get some of the definition counts while playing with groups. I see how they are dynamically added to the definition list when modified(which I actually think is kinda cool). Therefore, I don't see this affecting groups at all.

1) This modification to the parsing process could be tolerable, however it would have to be in conjunction with information about how the process affects SU workflow.

2) If it were possible, making this addition a toggle condition would allow maximum SU compatibility for those who want it, or speed if someone is willing to adapt their workflow. At the moment, I would consider the default behavior to still retrieve each component individually. That way, someone would have to toggle it on themselves, and therefore would know what will happen. In the future, this could transition to having the "fast parsing" on by default, but that's another discussion for another time.

3) The real advantage of this change in parsing would also require some additional tools to A) identify components that may not be compatible and B) adjust those components (either by reverting back to the original instance or making a new one) to work better with the parsing process. I could see a tool potentially identifying these issues as they are created (ie apply a material to a component and a dialog box popped up saying "this may not be good when u render")


I really wish some one else other than you two would chime in on this. I really appreciate your comments and responses, but I was hoping for a wider cross section of the VfSU community. Just 2 users doesn't seam like much of a consensus

[thomthom]

Re: Parsing, Components, and Workflow

I really wish some one else other than you two would chime in on this. I really appreciate your comments and responses, but I was hoping for a wider cross section of the VfSU community. Just 2 users doesn't seam like much of a consensus

I'll drop a note on the SCF forum and ask for feedback of the V-Ray users there. There's a few active members there which I don't think usually travel these forums.