A new Windows App is available from our Beta Test Page.

Compared to the 4.23 Beta App this one fixes a bug in memory access that might be responsible for some client errors. Also the "symbol store" debugging feature should work now.

For the curious: You can provoke a "client error" (Breakpoint) by putting a file named "EAH_MSC_BREAKPOINT" into the BOINC directory (remember to remove it after testing!).

BM

A new Windows App is available from our Beta Test Page.

Compared to the 4.23 Beta App this one fixes a bug in memory access that might be responsible for some client errors. Also the "symbol store" debugging feature should work now.

For the curious: You can provoke a "client error" (Breakpoint) by putting a file named "EAH_MSC_BREAKPOINT" into the BOINC directory (remember to revo it after testing!).

BM

Loaded and crunching ... 34 hours of work estimated on the Athlon 3200+. Wait and see :)
____________

Installed on two hosts.

I'm getting lots of x-platform-validation problems now, e.g. here http://einstein.phys.uwm.edu/results.php?hostid=962477, the series of workunits on that host seems to be a good testcase for the validation problem :-(.

Not sure whether it makes sense to continue crunching under Linux with the known issue of uninitialized variables at the moment.

CU

BRM
____________

Installed on two hosts.

I'm getting lots of x-platform-validation problems now, e.g. here http://einstein.phys.uwm.edu/results.php?hostid=962477, the series of workunits on that host seems to be a good testcase for the validation problem :-(.

Not sure whether it makes sense to continue crunching under Linux with the known issue of uninitialized variables at the moment.

CU

BRM

Are you saying that if a Linux host are paired up with any Windows OS that the Linux result is guaranteed to be declared "invalid"? I could've swore that I had one result validate against Linux some time ago, but I've been paired with nothing but other Windows versions lately.

Edit: It must be something specific with either the Linux or the Windows installation that is causing the issue, because not everything fails. I guess it could also be the frequency range of the result. Don't know. I'm currently watching FalconFly's results for comparisons against my processor. Though he said that he wouldn't be really processing a lot of stuff until this week, this WU of his validated against a Windows XP Pro host...

Are you saying that if a Linux host are paired up with any Windows OS that the Linux result is guaranteed to be declared "invalid"? I could've swore that I had one result validate against Linux some time ago, but I've been paired with nothing but other Windows versions lately.

Edit: It must be something specific with either the Linux or the Windows installation that is causing the issue, because not everything fails. I guess it could also be the frequency range of the result. Don't know. I'm currently watching FalconFly's results for comparisons against my processor. Though he said that he wouldn't be really processing a lot of stuff until this week, this WU of his validated against a Windows XP Pro host...

My Linux box just got 304.31 credits against a Windows box. Never had a validation error on my PII Deschutes running SuSE Linux 10.1, BOINC 5.8.17. Same in SETI and QMC.
Tullio

I'm pretty sure that the cross-platform validation problem neither has to do with the memory access bug we fixed, nor with a particular machine.

IMHO it is rather a problem of numerical differences between libraries or compilers that is triggered only by certain Workunits. It may be the actual frequency value or the data (files), e.g. a certain type of noise in particular frequency bands.

We're working on it.

BM

I remember that, at Area Science Park in Trieste, my MIPS 6000 minicomputer, running UNIX System V, gave results that were different from those obtained on a SUN workstation running SUNOS. But I still don't know why.
Tullio
____________

I'm pretty sure that the cross-platform validation problem neither has to do with the memory access bug we fixed, nor with a particular machine.

IMHO it is rather a problem of numerical differences between libraries or compilers that is triggered only by certain Workunits. It may be the actual frequency value or the data (files), e.g. a certain type of noise in particular frequency bands.

We're working on it.

BM

Some frequencies really seem to be more affected than others, I had a series of WUs lately where I had a 50 % failure rate (!), and a single result I investigated contained as many as 100 mismatches between Windows & Linux in the final "toplist" containing the 10000 most promising items.

Some frequencies really seem to be more affected than others, I had a series of WUs lately where I had a 50 % failure rate (!), and a single result I investigated contained as many as 100 mismatches between Windows & Linux in the final "toplist" containing the 10000 most promising items.

A) Where do you go to see that kind of stuff?

B) Where do you come up with the time to do that analysis? ;-)

Are you saying that if a Linux host are paired up with any Windows OS that the Linux result is guaranteed to be declared "invalid"? I could've swore that I had one result validate against Linux some time ago, but I've been paired with nothing but other Windows versions lately.

Edit: It must be something specific with either the Linux or the Windows installation that is causing the issue, because not everything fails. I guess it could also be the frequency range of the result. Don't know. I'm currently watching FalconFly's results for comparisons against my processor. Though he said that he wouldn't be really processing a lot of stuff until this week, this WU of his validated against a Windows XP Pro host...

My Linux box just got 304.31 credits against a Windows box. Never had a validation error on my PII Deschutes running SuSE Linux 10.1, BOINC 5.8.17. Same in SETI and QMC.
Tullio

Hmmm... Yep, I knew there were hosts out there going along fine. Obviously something triggers the problem...but what? Is it perhaps a problem with the WU generator? Pure speculation...and I'm probably talking out of ignorance at this point... LOL

P.S. - I HATE REGULAR EXPRESSIONS!!!!! Been up all night working on something for school. I realize they are "powerful and all", but they are way too nerdy, IMO...

Regualr Expressions are cool, but the the syntax is fairly difficult to get used to.

A): Just disable network access when you know you have a "special" workunit,

It's not too difficult to re-run a WU on another BOINC installation as well (offline, so you don't sent something to the server, of course!!!)

so sometimes you can't help and just HAVE to find out, I guess.

I'm just a poor boy... I can't go getting other hardware... I might check into VirtualPC 2007 or VMware player...but not until after this class is over. Too much to tackle getting set up otherwise (got to get apache or something like it set up this week).

The cross-platform tests I performed were done on the same hardware, just to make sure. I used my regular Win XP and a Linux Live-DVD (Knoppix in this case). (insert DVD, boot , enjoy Linux). Zero installation effort, very nice.

I'm pretty sure that the cross-platform validation problem neither has to do with the memory access bug we fixed, nor with a particular machine.

IMHO it is rather a problem of numerical differences between libraries or compilers that is triggered only by certain Workunits. It may be the actual frequency value or the data (files), e.g. a certain type of noise in particular frequency bands.

We're working on it.

BM

This may sound like a silly question, but. . .

If there are differences between the various compilers and math libraries, how do we know which ones will give scientifically accurate results? And, have a lot of us been producing results that are worthless?

This may sound like a silly question, but. . .

If there are differences between the various compilers and math libraries, how do we know which ones will give scientifically accurate results? And, have a lot of us been producing results that are worthless?

Interesting questions indeed. Follow up question: One way to improve validation would be to inject simulated "Pulsar Signals" into the input data and verify that the clients find them. Are there any plans to do that in the future?

CU

BRM

Finished first result with 4.24. Seems a bit slower, but dunno. No provocation (yet). I'll do that with one of the next group that I get...

Interesting questions indeed.
...
BRM

Interesting questions indeed.
...
BRM

I have an "interesting question". I tried to get VTune up and working on my machine to answer it, but since it is an AMD processor, it squawked about the processor architecture...and on top of that, I have no idea how to use the blessed thing... The C++ DLL that I worked on was not a performance drag (credit card auth on tcp/ip usually happened very quickly), so it was never "tuned"...

Sooooo.....

What is the effect that happens when you "ABC" again? Is that working against the modf() -> ftol() change, or is there still some activity going to modf() despite the change, meaning there's another "buggy detection" different from the one that was already worked around, or is that string changing some other function?

Brian

This flag toggles the (usually) faster SSE2 codepath for several functions, not just modf. What Bernd did was to rewrite the code in the hot-loop so that it would no longer call modf but ftol, for which, in VS 2003, only one code path exists which is reasonable fast. The slow codepath will continue to be executed in the new Win apps, but no longer in the hot-loop, as I understand it, so the overall effect of "ABC"ing the app should be much smaller now.

This may sound like a silly question, but. . .

If there are differences between the various compilers and math libraries, how do we know which ones will give scientifically accurate results?

And, have a lot of us been producing results that are worthless?

Interesting questions indeed. Follow up question: One way to improve validation would be to inject simulated "Pulsar Signals" into the input data and verify that the clients find them. Are there any plans to do that in the future?

For the curious: You can provoke a "client error" (Breakpoint) by putting a file named "EAH_MSC_BREAKPOINT" into the BOINC directory (remember to remove it after testing!).

Oh, just what we need... provoking... ;-) I don't think we should provoke Einstein. He may roll over and declare E=MC3 :-O

For the curious: You can provoke a "client error" (Breakpoint) by putting a file named "EAH_MSC_BREAKPOINT" into the BOINC directory (remember to remove it after testing!).

Oh, just what we need... provoking... ;-) I don't think we should provoke Einstein. He may roll over and declare E=MC3 :-O

Well, this is just for testing getting the symbols from the symbol store, so only if you're really curious. It will happen right at the beginning, so shouldn't waste computing time. You should probably set the project to "no new work" before you try this, and "allow more work" after you removed the file in order not to trash too many results. The result should look like this result, in particular you should find "PDB Symbols Loaded".

BM

1st WU finished w/o incident, and valid. My host was the 3rd to finish, zeroing out the credits of a Linux box. I do hope it wasn't you again, Gary :-(. I was lucky to get away alive with it last time...

CU

BRM

____________

Hey Bernd, got some odd errors for my WU and wanted to check back:

27/06/2007 23:59:37|Einstein@Home|[error] einstein_S5R2 not responding to screensaver, requesting exit
27/06/2007 23:59:38|Einstein@Home|Task h1_0493.15_S5R2__281_S5R2c_0 exited with zero status but no 'finished' file
27/06/2007 23:59:38|Einstein@Home|If this happens repeatedly you may need to reset the project.
27/06/2007 23:59:38|Einstein@Home|Restarting task h1_0493.15_S5R2__281_S5R2c_0 using einstein_S5R2 version 424
28/06/2007 00:00:40||Suspending computation - user is active
28/06/2007 00:06:23||Resuming computation
28/06/2007 00:06:23|Einstein@Home|[error] einstein_S5R2 not responding to screensaver, requesting exit
28/06/2007 00:06:24|Einstein@Home|Task h1_0493.15_S5R2__280_S5R2c_0 exited with zero status but no 'finished' file
28/06/2007 00:06:24|Einstein@Home|If this happens repeatedly you may need to reset the project.
28/06/2007 00:06:24|Einstein@Home|Restarting task h1_0493.15_S5R2__280_S5R2c_0 using einstein_S5R2 version 424

First WU with 4.24 completed and validated OK even with WinXP paired with Darwin wingman. Running AMD XP2600+ WinXP against wingman with Intel core 2/T5600 Darwin 8.9.1.

Appears to be using the same datapack as pre-4.24 app with time decrease of 28500 sec (approx 24% better)...that's just under 8 hrs less time/WU. Nice improvement on the AMD/Windows penalty. CPU does NOT have SSE2 capability, only SSE.

BTW, Bernd, did you notice this message from Gary Roberts? http://einstein.phys.uwm.edu/forum_thread.php?id=5848&nowrap=true#70886

It seems the l1_* files never get deleted, slowly filling up the disks of hosts until the quota is reached, effectively shutting down work for Einsein@Home after some time. Could be responsible for some hosts dropping out of E@H.

I plan to make this App official in the next few hours, mainly to get a more useful info regarding the client errors. It looks like it will at least make things not worse than the current official App (apart from Intel-SSE2 machines, which will see a small penalty of around 7%, but caugth up by the AMDs).

BM

... I do hope it wasn't you again, Gary :-(. I was lucky to get away alive with it last time...

It turns out there are a number of issues that lead to these cross-platform validation problems, some of which have been addressed recently, some we're still digging for. Solving these problems will probably require both a new validator and a complete set of Apps. I am confident that we will have all these pieces together next week.

BM

.... I am confident that we will have all these pieces together next week.

That's really great news, thank you!! With those issues sorted out soon, and with the prospect of significantly optimised apps to follow, hopefully people will be encouraged to hang on a bit longer or even return if they had already left.

I understand 4.24 is now the official release.

Would someone post here in this thread suggested procedure for those of us who got on 4.24 during Beta (using app_info.xml) to back out? This would mean we'd get the next official ap change automatically.

Is it as simple as stopping Boincmgr, removing app_info.xml in the Einstein directory, and restarting Boincmgr? (Assuming all results in queue are already tagged as 4.24 because they were downloaded after the previous change?)

Thanks
____________

Is it as simple as stopping Boincmgr, removing app_info.xml in the Einstein directory, and restarting Boincmgr? (Assuming all results in queue are already tagged as 4.24 because they were downloaded after the previous change?)

The notice on the Home Page regarding 4.24 is dated June 13, 2007. You may want to fix it...

My first result with the Windows 4.24 app completed during the night. For a 340-cobblestone workunit, completion time went from 29.86 hours to 22.46 hours. So, a nice little speed-up.
____________

The notice on the Home Page regarding 4.24 is dated June 13, 2007. You may want to fix it...

We get some errors with exit code 0x40010004 from hosts running Windows Vista. Did anyone run this App successfully and reliably on Vista, or is it failing on all such machines? Any clues what precisely might be the reason of this error?

BM

Success on a Vista Machine (Mine)

____________

We get some errors with exit code 0x40010004 from hosts running Windows Vista. Did anyone run this App successfully and reliably on Vista, or is it failing on all such machines? Any clues what precisely might be the reason of this error?

BM

Hi!

I just saw this one: http://einstein.phys.uwm.edu/result.php?resultid=85401300 and wondered whether the runtime debugger thing should not provide more detail here. Maybe there's still a problem with it? Or maybe it wasn't able to recover from the stack overflow error reported there to be executed properly?

CU

BRM

____________

Hi!

I just saw this one: http://einstein.phys.uwm.edu/result.php?resultid=85401300 and wondered whether the runtime debugger thing should not provide more detail here. Maybe there's still a problem with it? Or maybe it wasn't able to recover from the stack overflow error reported there to be executed properly?

CU

BRM

Sorry I don't know the exact version myself. it needs to be some version that installs the dbghelp.dll in the BOINC directory. 5.8.x is definitely ok, 5.2 sounds too old, should be something in between where the change occured. Anyone knows the version for sure?

BM

Bernd, the BOINC Windows Debugger was introduced in the 5.2.X series but a newer version was introduced with the 5.4.X series. Rom posted some technical details on the Ralph forum here which may help. Dave
____________

I just saw this one: http://einstein.phys.uwm.edu/result.php?resultid=85401300

My two AMD boxes (an AMD Athlon 2200 and an AMD Sempron 2600) are both nearly 25% faster!
____________
Udo

It turns out there are a number of issues that lead to these cross-platform validation problems, some of which have been addressed recently, some we're still digging for. Solving these problems will probably require both a new validator and a complete set of Apps. I am confident that we will have all these pieces together next week.

BM

Bernd,

Hopefully, whilst I've got your attention, you might like to review this thread concerning stalled results. I've noticed this behaviour a few times now and i've recorded the result ID of my latest stalled result there.

The result in question was being crunched with the 4.17 Windows app. A little while after I kicked it back to life, I decided to test out my app_info.xml mods in order to speed up the completion of the result as much as possible by using 4.24 instead of 4.17. Even though my result was past the deadline, a third result had not been issued at that point. I hoped that I might be able to beat the system and keep the third result "unsent" :).

Although there was a 25%+ speedup of the final stages of crunching, I still missed out on stopping the third result being issued by just 37 mins.




____________
Cheers,
Gary.

When you get to the point of deploying the new validator and the new set of apps, are you intending to run a (perhaps short) beta test phase first, as you did with the 4.24 Windows app?

If you are, might I make a suggestion about the app_info.xml file that would accompany each test app? As you warn quite clearly on the beta test page, changing the app aborts any work in progress with a client error. However you can easily avoid this with a small modification to the app_info.xml file. If you are already fully aware of this and do not want to allow a change of app in the middle of a result, that is fine - no change is needed.

My thinking is that the beta test period could be kept shorter and the number of potential beta testers could be increased if people were allowed to "re-brand" the results in their caches so that they didn't have to abort or wait for their caches to drain or in any way disrupt their normal crunching patterns in order to participate in the test. I'm sure that people have done this in the past by editing their state files. I think it's much safer to do it through the app_info.xml mechanism.

Furthermore if you really want to switch the App version halfway through a result, see the sticky post on this subject. I can not guarantee that it will work at all, as e.g. the syntax of the checkpoint file might change between versions.

Taking the case of the transition from 4.17 to 4.24 as an example. Here there were desirable bugfixes and apparently no change in output syntax. It would be prudent therefore for any 4.17 "branded" results in a person's cache to be crunched by 4.24, rather than the old buggy app. This can be achieved very simply using a bit more intelligence built into app_info.xml. No dodgy editing of the state file is required at all.

Currently it looks like upgrading some server-side components (validator and workunit generator) may solve the problem and be the best choice, but we're still looking into this.

BM

Currently it looks like upgrading some server-side components (validator and workunit generator) may solve the problem and be the best choice, but we're still looking into this.

Wouldn't it be worthwhile to correct the uninitialized data problem in the Linux and Mac apps? As those were detected by compiler runtime checks, to me it sounds as if they were relevant.

How about the 0xc0000142 crash issues? I don't know if you got my email, as you haven't replied... I wish I knew more of what to help with, but that error is a vexing one...

Edit: BTW, SIGABRT still seems to come up for Linux. See this result.

How about the 0xc0000142 crash issues? I don't know if you got my email, as you haven't replied... I wish I knew more of what to help with, but that error is a vexing one...

How about the 0xc0000142 crash issues? I don't know if you got my email, as you haven't replied... I wish I knew more of what to help with, but that error is a vexing one...

Is it still happeneing with the new app?? I would have guesses that the majority of these bugs were secondary problems resulting in a failure to initialize the runtime debugger (which should now work).

How about the 0xc0000142 crash issues? I don't know if you got my email, as you haven't replied... I wish I knew more of what to help with, but that error is a vexing one...

Edit: BTW, SIGABRT still seems to come up for Linux. See this result.

Just had a 4.24 crap out about half way thorough its first result run with 4.24.

85487605

Looks like it failed on a routine task switch restart.

Alinator

Just had a 4.24 crap out about half way thorough its first result run with 4.24.

85487605

Looks like it failed on a routine task switch restart.

Alinator

I have noticed one of these as well. At first glance it seems to be the same situation as Alinator's. It happened on the third result since the switch to 4.24.

Before I saw Alinator's report, I had attributed this error to hardware problems. With a large number of older machines, I've run across quite a number of motherboards which have developed the "swollen capacitor syndrome". Being curious by nature and owning a good quality Weller soldering iron, I've attempted the repair of about 10 such motherboards. Until now, my success rate has been 100% since all such repaired systems are back in production.

The result linked above was crunched on a machine where about 8 caps were replaced. I've only replaced caps that are obviously swollen so there are still some original caps left. It has been running fine for about 2 months since the repair but has started locking up about once a day recently. I've been restarting it as required and it has been completing work without any client errors until now. So I don't really know if the client error was associated with more faulty caps or a problem with the 4.24 app. Alinator's seemingly identical error is making me wonder if it's the app.

This weekend I'll probably take the mobo out and see if I can spot any more dodgy caps. If so I'll replace some more of them and see if that cures the lockups. It'll also be interesting to see if I get any more client errors on that particular combination of hardware, 4.24 app, and particular frequency data file, once I cure the lockups.


____________
Cheers,
Gary.

Very strange: It restarts, finds the checkpoint-file (!), tries to open it but somehow can't (!), and exists with an error message that the checkpoint file isn't there at all ...

To answer the question below re 4.24 vs 4.17 with-w/o patch, I went ahead and did the 'ABC' patch and got the following results:

4.17 with 'ABC' patch applied yields approx 85.3k sec/WU
4.24 with no patch yields approx 90.9k sec/WU
4.24 with 'ABC' patch applied yields approx 88.3k sec/WU...about a 50 min. penalty/WU for running 4.24 vs 4.17 on this machine.

What is the effect that happens when you "ABC" again? Is that working against the modf() -> ftol() change, or is there still some activity going to modf() despite the change, meaning there's another "buggy detection" different from the one that was already worked around, or is that string changing some other function?

Brian

Anyway...the effect of the "ABC" patch is that on AMD CPUs that supports SSE2, a global flag in the runtime lib is set differently. This flag toggles the (usually) faster SSE2 codepath for several functions, not just modf. What Bernd did was to rewrite the code in the hot-loop so that it would no longer call modf but ftol, for which, in VS 2003, only one code path exists which is reasonable fast. The slow codepath will continue to be executed in the new Win apps, but no longer in the hot-loop, as I understand it, so the overall effect of "ABC"ing the app should be much smaller now.

CU

H-B

Just to keep you updated of our plans, mainly regarding the cross-platform differences:
- Early next week (probably Monday) we'll issue a new validator that should make things easier for transition and probably fix some invalid results by itself
- After the new validator is in place, we'll issue a new set of Apps for public Beta Test (for all platforms) that incorporate the fixes accomplished so far. I'll keep on tracking problems and fixing bugs I find until the very last moment. The new Apps will also incorporate a new feature that we might need.
- If it turns out that we need this feature (using pre-calculated files instead of doing the calculations in the Apps to avoid platform differences there), we will issue new workunits (actually a new workunit generator) that will make use of this feature after the new Apps have been made "official".
- Once we got the validation working properly, I'll work on speeding up the computation in the Apps. The current code I plan to use for parts of the calculation btw. doesn't make use of neither modf() nor ftol() anymore but actually uses bit-operations to achieve something similar.

BM

Excellent news, and just in time to deal with the 'new' monster workunits (>= 630 credits) that would otherwise cause quite a bit of frustration if crunched for zero credits because of the cross platform validation issue.

As to performance, I was surprised to see that the new app with ftol instead of modf seems to be slightly *faster* at least on some modern SSE2-capable Intel (!!!) CPUSs. I know it's a tad-bit slower on my Pentium M, but I checked one of the top 3 computers (see link on E@H homepage) and there was no decrease in crunching performance when the switch happened.

CU

BRM




____________

My AMD 3500+ liked the 4.24, it went from about 38hr with 4.17 to ~28hr with 4.24 on WU from the same set of datafile, still waiting for my crunch partner to see if it is valid. If there is more to do too speed it up it´s great but I understand that the validation problem must be looked at first.

After the new validator is in place, we'll issue a new set of Apps for public Beta Test (for all platforms) that incorporate the fixes accomplished so far...

So is this a public beta test of 4.24?

E@H just automatically updated the existing 4.17 version when I connected.

I would concur with posting to news about this public beta phase if this is in fact the intention

So is this a public beta test of 4.24?

Bernd,
if the "precomputed skygrid" approach will be implemented, wouldn't that also offer an elegant way to handle different sizes of workunits again? The relativly long duration of the WUs seems to be a major issue with users.

Slower machines would crunch only (say) one hemisphere of the sky and faster ones the full sky.

Are there plans in this direction, maybe for S5R3?


CU

BRM


____________

Some of AKOS's computers have been found and they're running apps >5x faster than stock. Once the bugs are worked out, I think faster apps is the planned solution to the slow box problem.
____________

Just finished a wu with the beta version.

The cpu time was 59 hours 38 min. 12 sec ..

I'm one of those people running windows 2000 on a pentium 4 M -
where the sse2 set is not recognized for some reason.

2007-06-30 12:49:56 [---] Processor: 1 GenuineIntel Intel(R) Pentium(R) 4 Mobile CPU 1.70GHz [x86 Family 15 Model 2 Stepping 4] [fpu tsc sse mmx]

Even running with without optimization, this seems like a long time.
Other projects have WUs that take 1.5 hours
WCG-Fight aids at home (faah) takes about 38 hours (if my memory serves..)

So, I'm in favor of a shorter WU.
I assume no errors reported. the error file was cleaned up when the results were uploaded - I assume.

QUESTION: HOW TO BACK OUT THE BETA VERSION? - Dlete the beta xml file & application?

Thanks,
Jay E.

4.24 is no longer beta. It was pushed out as the mainstream win/x86 app about a week ago.
____________

QUESTION: HOW TO BACK OUT THE BETA VERSION? - Dlete the beta xml file & application?

QUESTION: HOW TO BACK OUT THE BETA VERSION? - Delete the beta xml file & application?

Here is essentially the same question as yours. It was asked in this very thread some days ago and Bernd answered it immediately - see the very next post for the answer.

Just to keep you updated of our plans, mainly regarding the cross-platform differences:
- Early next week (probably Monday) we'll issue a new validator that should make things easier for transition and probably fix some invalid results by itself
- After the new validator is in place, we'll issue a new set of Apps for public Beta Test (for all platforms) that incorporate the fixes accomplished so far. I'll keep on tracking problems and fixing bugs I find until the very last moment. The new Apps will also incorporate a new feature that we might need.
- If it turns out that we need this feature (using pre-calculated files instead of doing the calculations in the Apps to avoid platform differences there), we will issue new workunits (actually a new workunit generator) that will make use of this feature after the new Apps have been made "official".
- Once we got the validation working properly, I'll work on speeding up the computation in the Apps. The current code I plan to use for parts of the calculation btw. doesn't make use of neither modf() nor ftol() anymore but actually uses bit-operations to achieve something similar.

BM

Now that the validator issue has been resolved, are we almost to the point of beta testing a new batch of apps?

The Win App 4.24 dropped nearly 40 WUs with 99 (0x66) Recursive error exit code on my computer in the early hours, but it works well now.
My computer automatically switched to an other project, because the daily quota was reduced and I didn't get new WUs.

The Win App 4.24 dropped nearly 40 WUs ....

.... but it works well now.

I'm currently waiting for some internal tests to finish ...

The Win App 4.24 dropped nearly 40 WUs ....

The straight 4.24 app or one you had optimised?

.... but it works well now.

What caused it to start behaving again do you think? New (different frequency) data file perhaps?

An exit code of 99 means that the App terminated due to a failing internal sanity check. There should be a dump of a "status structure", similar to a stack dump, at the end of stderr_out, indicating the check that failed.

The most common cause of an error of this type is present when the following lines are found at the end of the dump:
[...] Status code 3: Incorrect header in file
[...] function LALSFTdataFind, file SFTfileIO.c, line 270

This means that a data file the App is trying to access has a broken header signature. The md5 checksum of downloaded files is checked by the BOINC Core Client only after downloading, so it might be that at some later point the file went bad on the disk. The fact that it has "cured itself" might be due to that you recently got work that doesn't require this particular file anymore.

There is a chance, though, that something else (I don't know of yet) is going wrong during accessing the file (e.g. it is blocked by a virus scanner) that the boinc_fopen() function that we are using doesn't catch.

Akos, what are the last few lines of stderr_out of the results in question? What other tools accessing the filesystem (virus scanner, malware removal etc.) are you using?
BTW: Anyone knows if the standard Microsoft Malware removal tool has any influence on BOINC Apps?

BM

Akos, what are the last few lines of stderr_out of the results in question?

<core_client_version>5.4.11</core_client_version>

<message>

 - exit code 99 (0x63)

</message>

<stderr_txt>

2007-07-12 23:57:31.9531 [normal]: Start of BOINC application 'projects/einstein.phys.uwm.edu/einstein_S5R2_4.24_windows_intelx86.exe'.

2007-07-12 23:57:32.2968 [debug]: Reading SFTs and setting up stacks ... Level 0: $Id: HierarchicalSearch.c,v 1.170 2007/06/08 20:58:34 bema Exp $

	Function call `SetUpSFTs( &status, &stackMultiSFT, &stackMultiNoiseWeights, &stackMultiDetStates, &usefulParams)' failed.

	file HierarchicalSearch.c, line 677

2007-07-12 23:57:33.5937 [normal]: 

Level 1: $Id: HierarchicalSearch.c,v 1.170 2007/06/08 20:58:34 bema Exp $

2007-07-12 23:57:33.5937 [normal]: 	Status code -1: Recursive error

2007-07-12 23:57:33.5937 [normal]: 	function SetUpSFTs, file HierarchicalSearch.c, line 1250

2007-07-12 23:57:33.5937 [normal]: 

Level 2: $Id: SFTfileIO.c,v 1.123 2007/04/24 15:32:38 bema Exp $

2007-07-12 23:57:33.5937 [normal]: 	Status code 3: Incorrect header in file

2007-07-12 23:57:33.5937 [normal]: 	function LALSFTdataFind, file SFTfileIO.c, line 270

2007-07-12 23:57:33.5937 [CRITICAL]: BOINC_LAL_ErrHand(): now calling boinc_finish()

What other tools accessing the filesystem (virus scanner, malware removal etc.) are you using?

BTW: Anyone knows if the standard Microsoft Malware removal tool has any influence on BOINC Apps?

BM

BTW: Anyone knows if the standard Microsoft Malware removal tool has any influence on BOINC Apps?

The one that you get on Microsoft Patchday? I never had any problems with it, but I guess if it would cause problems, they would kind of "stick out" statistically because most people will execute this tool automatically on MS Patchday, which is always a Wednesday, right? Might be worthwile to group errors by weekdays.

As to file corruption, the MD5 checksums are in client_state.xml, right? So one could check unless the file is now already deleted.

Akos, can you dig out the checksums from client_state.xml and check your data files? There's probably a simple too for Windows that does this (I usually use md5sum from Cygwin).

Here's another client error that looks kind of interesting:

http://einstein.phys.uwm.edu/result.php?resultid=85564323

Computation stopped near the end with this message:

45172, 45173, 45174, 45175, 45176, 45177, 45178, 45179, 45180, 45181, 45182, 45183, 45184, c
45185, 45186, 45187, 2007-07-13 18:59:59.8281 [CRITICAL]: Required frequency-bins [-8, 8] not covered by SFT-interval [788941, 789228]

XLAL Error - LocalXLALComputeFaFb (LocalComputeFstat.c:534): Input domain error
Level 0: $Id: HierarchicalSearch.c,v 1.170 2007/06/08 20:58:34 bema Exp $
Function call `COMPUTEFSTATFREQBAND ( &status, fstatVector.data + k, &thisPoint, stackMultiSFT.data[k], stackMultiNoiseWeights.data[k], stackMultiDetStates.data[k], &CFparams)' failed.
file HierarchicalSearch.c, line 1019
2007-07-13 18:59:59.8281 [normal]:
Level 1: $Id: LocalComputeFstat.c,v 1.34 2007/06/09 22:11:24 bema Exp $
2007-07-13 18:59:59.8281 [normal]: Status code -1: Recursive error
2007-07-13 18:59:59.8281 [normal]: function LocalComputeFStatFreqBand, file LocalComputeFstat.c, line 207
2007-07-13 18:59:59.8281 [normal]:
Level 2: $Id: LocalComputeFstat.c,v 1.34 2007/06/09 22:11:24 bema Exp $
2007-07-13 18:59:59.8281 [normal]: Status code 5: XLAL function call failed
2007-07-13 18:59:59.8281 [normal]: function LocalComputeFStat, file LocalComputeFstat.c, line 342
2007-07-13 18:59:59.8281 [CRITICAL]: BOINC_LAL_ErrHand(): now calling boinc_finish()


Wingman has completed it's result successfuly, also with Windows version 4.24. Go figure..
____________

Thanks for drawing my attention back to this one.

Pretty weird, the "Required frequency-bins" should be positive integers, but "[-8, 8]" is listed in most of these errors. Looking into it...

BM

Thanks for drawing my attention back to this one.

Pretty weird, the "Required frequency-bins" should be positive integers, but "[-8, 8]" is listed in most of these errors. Looking into it...

BM

See here.

BM

I noticed that around the time Visual Studio .NET 2003 was released, the programmers at the Folding@home project noticed that their science code in their clients, which was written in hand-crafted assembly code and takes advantage of SSE and 3DNow!, were causing earlier revisions of Athlons with SSE to crash, while Pentium III's and Pentium 4's were not crashing on the same code. Therefore, Microsoft was forced to add this code to avoid writing code that crashes because no one knew at the time what was causing this behavior. Later on, somebody found out that the Folding@home code was using non-aligned memory accesses in its SSE code, and that was causing the Athlons to crash and was slowing the other chips down. Since almost no commercially-released compiler would be stupid enough to generate any code with the mistake of non-aligned memory accesses anyways, maybe it is time to upgrade the compiler to Visual Studio 2005, which might have the detection code removed because a compiler will never make the mistake that will cause the chip bugs to crash those Athlons.

Sometimes, it takes assembly coders to generate unusual code to detect chip bugs.

I noticed that around the time Visual Studio .NET 2003 was released, the programmers at the Folding@home project noticed that their science code in their clients, which was written in hand-crafted assembly code and takes advantage of SSE and 3DNow!, were causing earlier revisions of Athlons with SSE to crash, while Pentium III's and Pentium 4's were not crashing on the same code. Therefore, Microsoft was forced to add this code to avoid writing code that crashes because no one knew at the time what was causing this behavior. Later on, somebody found out that the Folding@home code was using non-aligned memory accesses in its SSE code, and that was causing the Athlons to crash and was slowing the other chips down. Since almost no commercially-released compiler would be stupid enough to generate any code with the mistake of non-aligned memory accesses anyways, maybe it is time to upgrade the compiler to Visual Studio 2005, which might have the detection code removed because a compiler will never make the mistake that will cause the chip bugs to crash those Athlons.

I noticed that around the time Visual Studio .NET 2003 was released, the programmers at the Folding@home project noticed that their science code in their clients, which was written in hand-crafted assembly code and takes advantage of SSE and 3DNow!, were causing earlier revisions of Athlons with SSE to crash, while Pentium III's and Pentium 4's were not crashing on the same code. Therefore, Microsoft was forced to add this code to avoid writing code that crashes because no one knew at the time what was causing this behavior. Later on, somebody found out that the Folding@home code was using non-aligned memory accesses in its SSE code, and that was causing the Athlons to crash and was slowing the other chips down. Since almost no commercially-released compiler would be stupid enough to generate any code with the mistake of non-aligned memory accesses anyways, maybe it is time to upgrade the compiler to Visual Studio 2005, which might have the detection code removed because a compiler will never make the mistake that will cause the chip bugs to crash those Athlons.

Hi!

Interesting, is there a source where one could read more about this?

Visual Studio 2005 indeed does no longer have this detection code, but anyways one would have expected MS to correct this "issue" in one of their service packs to Visual Studio 2003.

Very much has changed in the floating point department between VS 2003 and 2005, and you can bet that also some new bugs have been introduced, so changing the compiler always comes at a certain risk. Anyway, the long term plans seem to be to compile the Windows version under gcc as well once that works.


CU

BRM

The hot ticket if you want performance is the Intel ICC/IPP combination. It produces code that is about 40% faster than gcc on Intel and 25% faster on other cpu's. Supports Linux, Windows (icc called by VS is an option), and Mac.

I noticed that around the time Visual Studio .NET 2003 was released, the programmers at the Folding@home project noticed that their science code in their clients, which was written in hand-crafted assembly code and takes advantage of SSE and 3DNow!, were causing earlier revisions of Athlons with SSE to crash, while Pentium III's and Pentium 4's were not crashing on the same code. Therefore, Microsoft was forced to add this code to avoid writing code that crashes because no one knew at the time what was causing this behavior. Later on, somebody found out that the Folding@home code was using non-aligned memory accesses in its SSE code, and that was causing the Athlons to crash and was slowing the other chips down. Since almost no commercially-released compiler would be stupid enough to generate any code with the mistake of non-aligned memory accesses anyways, maybe it is time to upgrade the compiler to Visual Studio 2005, which might have the detection code removed because a compiler will never make the mistake that will cause the chip bugs to crash those Athlons.

Hi!

Interesting, is there a source where one could read more about this?

Visual Studio 2005 indeed does no longer have this detection code, but anyways one would have expected MS to correct this "issue" in one of their service packs to Visual Studio 2003.

Very much has changed in the floating point department between VS 2003 and 2005, and you can bet that also some new bugs have been introduced, so changing the compiler always comes at a certain risk. Anyway, the long term plans seem to be to compile the Windows version under gcc as well once that works.


CU

BRM