Win class performance, cause (NEW), and a test for all

[DR_LaRRY_PEpPeR]

Pete, anyone, it's really simple. The "browser window" test, of course, but forget that now that the "GUI Bench" program is available -- nothing but the core affected functions, no varying browser config, etc. Just open the .zip in your sandbox and run it!

Leaving the default number of runs, the CreateWindow part runs "instantly" UNsandboxed. It should take at least 1 second, probably even on the fastest systems (??), in a sandbox with default settings. If one happens to have some window class settings that degrade performance (for whatever reason?!), and/or MORE windows and stuff open, it will get worse, etc... That's for just 1,000 CreateWindow calls, so it adds 1ms PER call (usually measured in millionths!). Of course it takes 10 seconds, give or take, on my system with defaults, so that adds 0.01s PER call! (1 million in 1.3s */UNsandboxed)

I'd be willing to bet anyone's Sandboxie license that they'll see the same results. It's definitely a Sandboxie issue. I'd be curious to see anyone's results prove me wrong.

To get a taste of how close you could be to crippling performance (more windows, "bad" OpenWinClass), just add to your settings:

ProcessGroup=<Anything>,*.exe
OpenWinClass=<Anything>,*

Not to discount tzuk or anyone else (not "noticing," etc.), but now we all have an example program to highlight the problem. I've already said before that depending on how much other "junk" someone is running on their system slowing things down already, of course the overall relative difference can be less. I run NOTHING to slow stuff down. Heck, it's like even Sandboxie 3.76 chokes browser opening, etc. (but it's "just tolerable" and understandable). I'm used to almost INSTANT everything without sandboxing, so I'm just hoping to help get as close back to that experience as possible.




tzuk, I think this also nailed why toggling Work Offline in IE 6 freezes the browser for a bit (that I've been mentioning for a while). Toggling Work Offline does 30 CreateWindow calls (almost as many as opening a new IE window: 30-something). Since Sandboxie is adding ~0.01 per, that's 0.3s which is just about what I'm feeling. Obviously, it gets worse with more windows open. Using * (prefix) will freeze IE for a FEW SECONDS. Again, just verifying the pattern...

Firefox? Not as many CreateWindow calls, which explains why I never felt too much with it. Is it using resource-based dialogs or something?? Not sure how they make their window stuff with fewer CreateWindows (but I don't know that much about this stuff!), but as I said, I couldn't cause a slowdown with my resource-based dialog anyway.

The CreateWindow slowdown would also explain the delay I've always felt (default settings) with Common Open/Save dialogs, OE's Create Mail, etc.

API Monitor is REALLY slow like I said (didn't investigate at all) -- maybe it's more like those other super-slow programs others have reported?

Finally, another slow one is PowerArchiver (2011)... Launching? 100-something CreateWindow calls. Options > Configuration window? 200+ calls! Slow!


Everything fits the explanation. Why is CreateWindow so slow? (I can think of a few things that happen, but don't know what's taking so much extra time.) What is Sandboxie doing with all the CPU? Why do unrelated Open classes affect it?


BTW, I noticed in 3.76 that CreateWindow also gets slower as more windows are opened (it just starts out better). I probably did notice that a bit last fall. And instead of SbieSvc, running GUI Bench causes some CPU usage in sandboxed IE process!? Again, doesn't seem right (but nothing else affects it). Even 3.76's best CreateWindow case (125x slower) seems like too much!

[Peter2150]

Your missing my point. I haven't seen anything worth bothering to test. Maybe if I put my system under a horrendous load I might notice something, but since I don't it just isn't worth my time. Period.

[DR_LaRRY_PEpPeR]

What load...?! If by "horrendous load," you mean otherwise-idle CPU and a dozen windows open (not all sandboxed)?

The minimum anyone should see is what I see on 2 freshly-booted, nothing else installed, nothing else running systems.

Any regular system will have window creation with at least 1,000x more overhead. Period. And it doesn't take anything spectacular/"horrendous" for that to increase to 10,000x (as my main system shows).


Please, somebody report/show fast window creation results... Can't happen!

The point is, now tzuk has the real cause (finally, AFAICT), and a simple function/"program" to look at.

[Peter2150]

What load...?!



Please, somebody report/show fast window creation results... Can't happen!

The point is, now tzuk has the real cause (finally, AFAICT), and a simple function/"program" to look at.

Yes, but you are missing one vital point. My pc configurations have stayed constant with the exception of updating 3 security programs including SBIE. Going back to as far as 3.6 thru 4.05.02, I have not seen any change in system performance. I do shut my system down every night.

So for myself, and I suspect a lot others here, there are probably a lot of others here who frankly could care less about this. What I care about is that when I get a conflict with a piece of software I, and others use, that Tzuk finds a fix.

[tzuk]

DR_LaRRY_PEpPeR you keep insisting that I have to fix this problem, and I keep explaining why I don't think it is good use of my time.

Here it is once again: While benchmark numbers might show that version 4 is a bit slower, I am not seeing a practical difference in performance. Finally the version 4 code is reaching a point where it is working relatively well, I am not going to risk that by introducing changes that are designed to shave off a few milliseconds.

If you want me to fix the problem, please identify the component on your system that is making performance unbearable for you. If there is no such component and you are pointing out a difference of a few milliseconds, then as I explained in the paragraph above, I am not going to fix it.

[DR_LaRRY_PEpPeR]

Can you please stop referring to "a few milliseconds" so often? I would not ever notice that (not to mention it's well within a margin of fluctuation, etc.), much less ask you to spend time "improving" something that hasn't really been degraded. 3.76 doubled IE startup (200ms) -- again, understandable and acceptable. I get how things work. But this is a real problem. As I said, with default settings and my usual windows open, Sandboxie is adding 0.01s PER CreateWindow call -- how could that possibly be a "few milliseconds??"

(Also, I wouldn't say "insisting," I've just always been trying to report stuff to the best of my ability (very well, I think), to help, or help you (asking?), make Sandboxie as BEST as can be, in every way, now and forever. )

Have you ran the GUI Bench program now to see how it runs? Can you share your results, and observe how HORRENDOUSLY (good word Pete ) different OpenWinClass values affect it, strangely?

Simple request: Please optimize GUI Bench. It's running really, really, really slow! Or will you find something wrong with my code? Granted, I hardly know what I'm doing, but not really anything to mess up there!

Like I said, I wasn't mentioning this stuff for a bit, because there was nothing new to say, and living with it... Then the point of this topic was for the "concrete specifics," that I thought I found in the first post, right? And it turns out, I was actually right about those "problem classes." For whatever reason, they ARE slowing down unrelated CreateWindow calls big time. Like it or not, there's GOT to be something weird/wrong going on there, no?

Again, I understand if you have other "real" things to work on sooner, etc., but just to take a look to have some idea of what's happening and optimize it in the next months or so would be GREAT. It's just disappointing while it's operating this way, when it SEEMS if a few parts would be optimized it could be running AMAZING. BTW, I still can't figure out exactly WHY PerformanceTest's 2D Window Interface benchmark runs BETTER in Sandboxie 4 than UNsandboxed/3.76 -- really puzzling!

But yeah, I'm optimistic... I don't think there's a lot to risk or drastically change operation if there's just an inefficient way of doing SOMETHING right now (not a big deal to optimize, better algorithm, etc.). Just a faster way to run the same logic/functionality...?

I guess there's no code/code overview/pseudo-code to share, or anything where I could help with any optimization? I just can't see, what in the world is going on with CreateWindow, that doesn't seem to happen with resource-based dialogs, that's making it 1,000-100,000x or more slower? I mean, just nothing. What lookups, lowercasing, matching, searching, etc. operations could take so long? I SURE you can find some explanation now that we know which function to check. If you've never checked anything, maybe just a simple thing you overlooked...? *shrug* It would be interesting to hear what the deal is with CreateWindow.


I've truly done everything now (as a n00b wannabe Win coder ) to get you precise details of the issue(s). I wish you could've checked function calls with IE, for example, months ago and spotted the slowdown sooner... I think (and hope for you/us) that it would take care of the other performance problems, more severe than mine, that others reported.


You keep insisting that there's a "component" on my systemS, and I keep explaining that there's not. If you insist, fine, but can you please help me identify this magical mystery thing when nothing has been installed or changed on 2 systems? I don't like that explanation/excuse, and don't like when anyone/companies/support tries to blame my stuff. With everything, it seems like all I'm ever doing is finding bugs in stuff (one reason I avoid software as much as possible), and/or fixing other people's code/stuff...


Please see what happens with GUI Bench on your system(s), and hopefully you see what I or anyone will see with the SbieSvc process(es).

I am seeing cores * 2 threads running in the main SbieSvc with equal CPU usage and context switches... That seems odd, what's going on there? Are those your threads or an indirect Windows thing? (Starting in kernel32.dll, but I don't understand much thread stuff...) But why multiple running threads? I don't see how anything could be parallelized from the CreateWindow call?


I was hoping for some good news after isolating the cause, so I'm optimistic that we can get CreateWindow flying.

[tzuk]

I say insist because it seems you seem to be spending an awful lot of time on something that I consider a non-problem.

Maybe you have some stuff running in the sandbox that is trying to install hooks on CreateWindow and other windowing operations, via the SetWindowsHookEx API. In version 3.76 the hook acts in the normal way and runs in the context of the process where the interesting event occurs. In version 4 the hook runs in the context of the process that installed the hook. Such a context switch is relatively expensive and perhaps that plays a part in the slowdown you perceive.

(Note that if the hook is installed by a process outside the sandbox then the above does not apply.)

I am suggesting this because I remain of the opinion that if you see a noticeable performance impact then it must be caused by something in your system, and I would be happy to take a look at it once we identify what it is.

[DR_LaRRY_PEpPeR]

Don't worry, I'm not spending "too much" time on this (of course I'd like to have not spent any, but...). I had never made a functional GUI program, so I learned a bit (basics!) creating the GUI Bench. That's fine, good actually, since I needed to learn some things for another functional project I'm going to do! (I had been putting it off. ) So no biggie. But yeah, I get it.

You know I posted a new GUI Bench PROGRAM, right? I'm not talking about the browser window test thing. Will you run it? If you can see the differences I posted, then we don't have to talk about my specific systems.


No, nothing hooking inside the sandbox -- just standard browsers/OE running. GUI Bench only thing running sandboxed for the results I posted.

On the main system, I have AutoSizer and 4t Tray Minimizer (bleh) outside the sandbox. Of course I expect them to have some impact on stuff. Stopping them makes a SMALL difference both inside and outside a sandbox -- ~same relative difference, therefore same relative sandbox impact all things being equal.

No hooking stuff installed on the other systems. I think if I open 10-12 IE windows on clean XP laptop, CreateWindow will slow down almost like main system.


Besides: 1) Hooks wouldn't explain why e.g. OpenWinClass=Tooltips_class32 makes CreateWindow like 3x slower. 2) Aren't dialog boxes also affected by hooks? (I have no idea.)

[Peter2150]

@Tzuk

I agree with you a thousand percent.

@Dr_LaRRY PepPer

So you posted a nbew GUI Bench Program. But what you don't want to seem to understand is by no participation here, most people find that Sandboxie is having no adverse impact on them. Ergo re your program, they really just don't care.

Lets end this.

[DR_LaRRY_PEpPeR]

No, why end this now? I just pinpointed the cause! I reported it months ago, nothing was really ever looked into, so I figured it out myself, finally, with just a little effort. If you're not a developer ("agreeing," when you don't even understand what we're talking about?), have some super fast new computer (2 are 5 years old, doubt I'd see much difference on a new one; maybe Sandboxie, not in general), or are running so much other "crap" software (remember, I'm NOT, but am accused of it!) that you don't notice any difference, fine, but I'm not sure why you're objecting.

Again, if you agree with tzuk, post your fast GUI Bench result. Nobody has shown that either... I'm just waiting to hear tzuk's results. (His couple browser results already show the same pattern.)

Why not? I know you said it's not worth your time, yet you're posting here? Take 30 seconds and run the program! If it's fast, you and tzuk are right. If it's not, I'm right. Are you then going to try to tell me that my programs are somehow calling CreateWindow "too much?" As if they're not normal, regular programs? No, it's very simple, CreateWindow is slowed down by X amount, and that amount adds up when programs are operating normally.

When X is 0.01s and a program calls CreateWindow 200 times, it's delayed by 2 seconds! What's so hard to understand about that?

Pete, are you telling me OpenWinClass of * (prefix) is something that we aren't supposed to use? I don't [use it], but you're saying it's an illegitimate setting? Have you tried these settings and felt "horrendous load?"

ProcessGroup=<Anything>,*.exe
OpenWinClass=<Anything>,*

PowerArchiver (2011; 2012 prob same) takes 25 seconds to launch! 50 seconds to bring up Configuration window! 100-ish vs 200+ CreateWindow calls: Duh! Notice the correlation? DO NOT tell me I'm wrong. Do you get it? Do you know what over 120,000x slower means...?


Sandboxie's settings/processing is WRONG, period! Nothing else can explain why adding OpenWinClass=Tooltips_class32 makes it slower for NO reason, except something very inefficient/wrong.

I've only ever posted correct information as a "blind outsider." Frankly, it seems like tzuk doesn't even know what's affecting what, since everything as it's been explaining to me before is WRONG, period.


We all just want Sandboxie to be as best as can be, right? Who knows, maybe in the future, you would somehow be affected by this (have you seen other posts? some were deleted!).

[tzuk]

I just tried PowerArchiver, it comes up in about three seconds, in or out of sandbox. Configuration window seems to open slower than outside the sandbox -- I'll give you that -- about 2-3 seconds in the sandbox, about 1 out of the sandbox.

Now, I can certainly understand making a big deal that things are running at 25 and 50 seconds, but that's just not the experience for me. You keep asking me to spend time to fix things that are running at most one second slower for me, and you think that is going to fix your 50 second problem? Not likely. It has to be something else that is unique to your system (all three of them even).

[DR_LaRRY_PEpPeR]

25-50 seconds is with * (prefix), I hope you realize that. Example of the worst case. And you can't explain why that's SO slow? (10x or more than defaults.)

Your delay in the sandbox (Configuration) should be all caused by CreateWindow. You can see that, as I did, with e.g. API Monitor, which itself is HORRIBLY slow sandboxed with default settings even, as I said.

Why are you running PowerArchiver (it was an extreme example for Pete) instead of GUI Bench, which will show you the difference with CreateWindow?!

It seems like you just keep avoiding this! Why? I've been waiting to see what you report with THAT since I posted it. Otherwise with other program stuff I explain, it's just excuses.

I thought you'd be satisfied that I'm showing you exactly what to look at? After all the months just wondering, I finally found it, and...


1) Is it not simple enough to run GUI Bench and note the time (to yourself or here)?

2) Add completely unrelated OpenWinClass settings ("problem classes," $:explorer.exe, whatever), up to * (prefix), and note how much slower it gets.


Again, regarding 2, you're going to come back here and try to tell me it's something on my systems? That is simply not possible. It's ONLY Sandboxie stuff that changes. And I'm telling you that magnified slowness is just a result of whatever else is going on with default settings...

*sigh* Will you PLEASE run the GUI Bench? *I* saw the results with a quick CLI test, and made the GUI so it'd be dead simple for anyone else to check (plus I needed a window handle for the first window access test).

[tzuk]

I am running PowerArchiver because it is a real world use case. With OpenWinClass=*tooltip* I am still seeing the same times, that is around 3 seconds for start up and for configuration window.

* * *

As for your secondary benchmark I am not sure you fully understand the concept of the GUI proxy process. The idea is that programs in the sandbox can only directly access handles for window objects in the same sandbox. For anything else it has to go through the GUI proxy process. That is usually not a big deal because most programs do not spend all their time looking at windows handles that don't belong to them.

So then you craft a benchmark that looks at handles not in the sandbox (invalid handle in your case), and then you say - look, it's slow! Of course it's slow. But in practical terms it doesn't matter; and there is no guarantee that it can be improved much; and there is little incentive for me to try to make it better when it hardly matters in the big picture.

* * *

As for CreateWindow, I am testing a change that your GUIBench says makes a difference from 40 seconds (for the 10^4 calls case) to 10 seconds. Hopefully that will do.

[DR_LaRRY_PEpPeR]

As for your secondary benchmark I am not sure you fully understand the concept of the GUI proxy process. The idea is that programs in the sandbox can only directly access handles for window objects in the same sandbox.

Yes, I do understand. (Nothing else to say about this right now...)

For anything else it has to go through the GUI proxy process. That is usually not a big deal because most programs do not spend all their time looking at windows handles that don't belong to them.

So then you craft a benchmark that looks at handles not in the sandbox (invalid handle in your case), and then you say - look, it's slow! Of course it's slow. But in practical terms it doesn't matter; and there is no guarantee that it can be improved much; and there is little incentive for me to try to make it better when it hardly matters in the big picture.

Craft? Come on. I realize exactly what's going on (I just thought it seemed a little too slow, anyway).

As I completely explained in my first post (seems you never saw), that "invalid" benchmark I "crafted" was the VERY first thing I came up with when I started trying SOME sort of function (with an Explorer handle)... And you make it sound like I picked it to make Sandboxie look bad...

I said, before I was going to post, I realized that test did NOT make the difference I thought, and had to find the real cause -- CreateWindow later. I simply left that first "access" test there because it already was.

It's just amazing how I explain every detail, it's not even seen (seemingly), and then get complaints about leaving an additional test in! Did I ever mention the "access" benchmark in these later posts?

As for CreateWindow, I am testing a change that your GUIBench says makes a difference from 40 seconds (for the 10^4 calls case) to 10 seconds. Hopefully that will do.

Well... that was quick! Can't believe you could make a change that fast. (Assuming you just recently ran.) Curious about that, of course...

It would be nice to get it down to around 100x slower (should be OK speed, and leaves a lot of extra time to do what it needs?). 4x improvement doesn't make that, but I'm optimistic like I said.

[tzuk]

Alright, I withdraw my objections to your invalid handle test, and I apologize.

About CreateWindow, very little code there was candidate for being a time sink in the first place.
One thing was going through the SbieSvc GUI process, for window handles in the sandbox, so I changed that now.
Hopefully that change goes well without any surprise problems.

Here, you can try the revised DLL for version 4.05.03. 32-bit only.
http://www.sandboxie.com/SbieDll.dll