except for porting to new platforms DIY apps aren't wanted. Akos was distributing apps for S4 that were several times faster than the stock app but was asked to stop doing so at the start of S5 due to concerns about the provable correctness of the results.
The compiled apps you will find following those links are currently (at the time I write this) *not yet* release candidates. They will not work with the new S5R4 workunits yet. The final official apps for S5R4 will be released next week AFAIK.
I did exactly this (dual apps running side by side) for the previous transition (S5R2/S5R3). I documented everything in great detail in this old thread and people should go read that carefully and understand what is involved if they intend to attempt to do it for themselves.
Quote:
I think about somethin like:
....
That's fine in principle. You can't really finalise things in advance even if you think you know the app names and versions now because there is no guarantee whatsoever that anything to do with names, version numbers, numbers of files, etc, is anywhere near static yet. The responsible thing to do is wait for the official release and then publish a suitable app_info.xml file once all the details are known and you have personally tested it to work properly. The last thing we want is for people to screw up whole cache-loads of work because of a rush to get an ill-tested file published.
There will be S5R3 work available for weeks to come yet so there really is no rush. I intend to publish working files but only after I'm quite sure it works properly on a test machine.
As archae86 pointed out that people who have been using the anonymous platform mechanism (app_info.xml) do really need to be clear about how they intend to do the transition. Many will use the safe and simple procedure he published and that is fine. There is no rush. When you get sick of doing resends or they seem to be running out, just set NNT and allow your cache to drain. Report the final results, delete app_info.xml and then restart BOINC to join the new run. The only disadvantage with this is exactly what archae86 mentioned - you may still get future resend work sent to you anyway and in that case you would be using the old stock app rather than the best performing app of your choice. The only way to guarantee using the old app of your choice is to continue using a "dual app capable" app_info file.
For that smaller proportion of people who have special needs or want to do something a bit more involved, there are different approaches that you can take.
One of those is to make yourself "dual app capable using apps of your choice" along the lines of the example file that I've documented previously or the one published by ziegenmelker here. Go read the linked old thread if you want all the details. My intention is to publish suitable app_info files to do this once all the new details are known.
Another special use is actually to prevent your hosts from receiving any further resend work if you find that these resends are consuming too much bandwidth. Even though the newly released S5R4 app will be a stock app, you can still run it under the anonymous platform mechanism if you wish. By doing so, you declare to the server that you can only receive work for this app. In other words, you wont get any further resend work for the old app from that point on. A bit anti-social if you only have one or two hosts but maybe a necessity if you have a couple of hundred and have costly bandwidth limits :-).
For the file names have a look at the download page. ;-)
It was actually ziegenmelker who said that - but who's quibbling :-).
Quote:
and, a while ago, Gary Roberts said:
Quote:
So what you could do as soon as the new run is "live" and the apps are available (but you are still crunching the old) is simply download copies of the new app files and place them in your E@H project directory, edit your copy of app_info.xml there, and then stop and restart BOINC. From that point on you are "new and old app ready" and will be announcing same to the server every time you contact.
And where is that, pray tell?
It appears that many (perhaps all??) of the previous stock apps are in the download directory. If you have a machine that is using the stock app, the URL where that app is available should be listed in client_state.xml, in the same way that URLs for sun and earth files are also listed. I don't think I actually have a single "stock app" machine where I can check this. If you look at all the URLs for data files, skygrids etc, you will see they come from subdirectories of the download directory. The new apps will be in there somewhere once they are released. I'm sure some kind soul will advise us when the new app first hits the streets and so it should be relatively simple to grab a private copy in advance of actually needing to use it.
Who is the first/next building his own einstein app? :-))
cu,
Michael
Are you putting up your hand?? :-).
Are we going to add ziegenmelker to the illustrious list of names such as Alex Kan et al, Simon Zadra et al, Jason Gee et al, JDWhale, etc, etc, who have done this sort of thing in the Seti world?? :-).
Actually, Bernd and others behind the scenes (with notable user support from people like Akos and Bikeman) seem to be doing a pretty decent job of tweaking the apps for best performance so perhaps it will be difficult for others to make further significant improvement.
We tend to lose sight of the fact that when S5R3 started at the end of September last year, the initial indications were that the run was going to take well in excess of a year (maybe even 1.5 years) with the apps available at that time. It also took quite a while for significant speed improvements to be made so that the fact that we are now finishing up in about 310 days only is quite amazing. The improved performance of the apps has played a very significant part in this. Of course the recent growth in active hosts certainly helps as well.
One other point of interest. I was still receiving significant S5R2 resend work at the end of October last year, more than a month after S5R3 had started. I guess it's going to be the same for at least the next month this time as well.
Actually, Bernd and others behind the scenes (with notable user support from people like Akos and Bikeman) seem to be doing a pretty decent job of tweaking the apps for best performance so perhaps it will be difficult for others to make further significant improvement.
We tend to lose sight of the fact that when S5R3 started at the end of September last year, the initial indications were that the run was going to take well in excess of a year (maybe even 1.5 years) with the apps available at that time. It also took quite a while for significant speed improvements to be made so that the fact that we are now finishing up in about 310 days only is quite amazing. The improved performance of the apps has played a very significant part in this. Of course the recent growth in active hosts certainly helps as well.
Probably it is worth mentioning that there will be likely some law of diminishing returns for effort with optimisations. There will be some irreducible computational complexity for a given quality of output/analysis. Choice of algorithm, assembly level tweaking, end-runs on compiler quirks etc are all going to impact - but not with endless runtime reductions.
Quote:
A bit like land speed records. They were broken initially in leaps and bounds but later on with lesser and lesser increments and at considerably greater investments as the underlying physical limitations were approached. The last one ( breaking the speed of sound ) had a total engine power about five times that of a high-performance military jet! Over ~ 2/3 of which contributed to shoving the 'car' into the ground to keep it NOT airborne.
Of course if major aspects of the problem change then one is looking at a renewed/different scope.
Cheers, Mike.
I have made this letter longer than usual because I lack the time to make it shorter ...
... and my other CPU is a Ryzen 5950X :-) Blaise Pascal
If we look back on the S5R3 run, the speed of the app increased by a factor of 2 to 3. I also think this will be difficult to repeat in S5R4, unless you make changes on the algorithmic level, which is both difficult and risky.
But speed is not everything. Porting the app to additional platforms is another area where volunteer contributions might be useful & welcome (64 bit code, and the Cell or Sparc CPUs come to mind, as well as different GPUs, for which some work has already been done, tho).
However, that's my personal opinion only. I guess Bernd will have more time to expand on this once S5R4 is done.
Actually, Bernd and others behind the scenes (with notable user support from people like Akos and Bikeman) seem to be doing a pretty decent job of tweaking the apps for best performance so perhaps it will be difficult for others to make further significant improvement.
We tend to lose sight of the fact that when S5R3 started at the end of September last year, the initial indications were that the run was going to take well in excess of a year (maybe even 1.5 years) with the apps available at that time. It also took quite a while for significant speed improvements to be made so that the fact that we are now finishing up in about 310 days only is quite amazing. The improved performance of the apps has played a very significant part in this. Of course the recent growth in active hosts certainly helps as well.
Probably it is worth mentioning that there will be likely some law of diminishing returns for effort with optimisations. There will be some irreducible computational complexity for a given quality of output/analysis. Choice of algorithm, assembly level tweaking, end-runs on compiler quirks etc are all going to impact - but not with endless runtime reductions.
I asked the same question some time ago and mentioned that people would continue to want SSE3, SSE4, etc, etc, etc... but I think it is fair to say that SSE2 seems to have enough performance advantage to be implemented.
As for the active hosts, I would imagine that the misfortunes over at Cosmology will contribute to higher participation here. I know that I'm going to be reevaluating my contribution there, given all the problems with the servers, the applications, and finally the fact that as viewed from my system, it now pays out credit on the level of LHC, which was already amongst the lowest BOINC-wide.
Quote:
Quote:
A bit like land speed records. They were broken initially in leaps and bounds but later on with lesser and lesser increments and at considerably greater investments as the underlying physical limitations were approached. The last one ( breaking the speed of sound ) had a total engine power about five times that of a high-performance military jet! Over ~ 2/3 of which contributed to shoving the 'car' into the ground to keep it NOT airborne.
The military jet type I've always been fascinated by is the A-12 / SR-71. It is well noted that the subsonic performance is terrible, with the plane having to dive to be able to break the sonic barrier. Once supersonic though, nothing (we know of) can match it. If a missle is fired at it, the standard procedure is to just accelerate and/or accelerate and climb...
I asked the same question some time ago and mentioned that people would continue to want SSE3, SSE4, etc, etc, etc... but I think it is fair to say that SSE2 seems to have enough performance advantage to be implemented....
.... The military jet type I've always been fascinated by is the A-12 / SR-71. It is well noted that the subsonic performance is terrible, with the plane having to dive to be able to break the sonic barrier. Once supersonic though, nothing (we know of) can match it. If a missle is fired at it, the standard procedure is to just accelerate and/or accelerate and climb...
Well, ideally we could have a broad suite of ultra-fine tuned highly optimised app versions, one for each known possible target processor variant on the planet. Maybe that'll be in place by the time we're doing S7R1 and beyond! :-)
I note the scheduled outage this w/end, at a good time b/w runs to drop upgrades in. I wonder what sort of new/hot gear is going in?
Cheers, Mike.
( edit ) Might I also congratulate the project management, developers and all manner of testers and contributors [ and mods :-)] for a pretty seamless run. Their dedication, responsiveness and attention to detail has yielded a very well performing DC project. Touch wood!
I have made this letter longer than usual because I lack the time to make it shorter ...
... and my other CPU is a Ryzen 5950X :-) Blaise Pascal
Well, ideally we could have a broad suite of ultra-fine tuned highly optimised app versions, one for each known possible target processor variant on the planet. Maybe that'll be in place by the time we're doing S7R1 and beyond! :-)
I note the scheduled outage this w/end, at a good time b/w runs to drop upgrades in. I wonder what sort of new/hot gear is going in?
Cheers, Mike.
LOL...
Regarding the first remark... Good One! :-D
Pertaining to the second... There goes their continuous uptime record! Still, pretty impressive though. ;-)
except for porting to new
)
except for porting to new platforms DIY apps aren't wanted. Akos was distributing apps for S4 that were several times faster than the stock app but was asked to stop doing so at the start of S5 due to concerns about the provable correctness of the results.
Hi, The compiled apps you
)
Hi,
The compiled apps you will find following those links are currently (at the time I write this) *not yet* release candidates. They will not work with the new S5R4 workunits yet. The final official apps for S5R4 will be released next week AFAIK.
CU
Bikeman
RE: Is there a way to edit
)
Yes, as recommended earlier in this thread.
I did exactly this (dual apps running side by side) for the previous transition (S5R2/S5R3). I documented everything in great detail in this old thread and people should go read that carefully and understand what is involved if they intend to attempt to do it for themselves.
That's fine in principle. You can't really finalise things in advance even if you think you know the app names and versions now because there is no guarantee whatsoever that anything to do with names, version numbers, numbers of files, etc, is anywhere near static yet. The responsible thing to do is wait for the official release and then publish a suitable app_info.xml file once all the details are known and you have personally tested it to work properly. The last thing we want is for people to screw up whole cache-loads of work because of a rush to get an ill-tested file published.
There will be S5R3 work available for weeks to come yet so there really is no rush. I intend to publish working files but only after I'm quite sure it works properly on a test machine.
As archae86 pointed out that people who have been using the anonymous platform mechanism (app_info.xml) do really need to be clear about how they intend to do the transition. Many will use the safe and simple procedure he published and that is fine. There is no rush. When you get sick of doing resends or they seem to be running out, just set NNT and allow your cache to drain. Report the final results, delete app_info.xml and then restart BOINC to join the new run. The only disadvantage with this is exactly what archae86 mentioned - you may still get future resend work sent to you anyway and in that case you would be using the old stock app rather than the best performing app of your choice. The only way to guarantee using the old app of your choice is to continue using a "dual app capable" app_info file.
For that smaller proportion of people who have special needs or want to do something a bit more involved, there are different approaches that you can take.
One of those is to make yourself "dual app capable using apps of your choice" along the lines of the example file that I've documented previously or the one published by ziegenmelker here. Go read the linked old thread if you want all the details. My intention is to publish suitable app_info files to do this once all the new details are known.
Another special use is actually to prevent your hosts from receiving any further resend work if you find that these resends are consuming too much bandwidth. Even though the newly released S5R4 app will be a stock app, you can still run it under the anonymous platform mechanism if you wish. By doing so, you declare to the server that you can only receive work for this app. In other words, you wont get any further resend work for the old app from that point on. A bit anti-social if you only have one or two hosts but maybe a necessity if you have a couple of hundred and have costly bandwidth limits :-).
Cheers,
Gary.
RE: Googloo said:RE: For
)
It was actually ziegenmelker who said that - but who's quibbling :-).
It appears that many (perhaps all??) of the previous stock apps are in the download directory. If you have a machine that is using the stock app, the URL where that app is available should be listed in client_state.xml, in the same way that URLs for sun and earth files are also listed. I don't think I actually have a single "stock app" machine where I can check this. If you look at all the URLs for data files, skygrids etc, you will see they come from subdirectories of the download directory. The new apps will be in there somewhere once they are released. I'm sure some kind soul will advise us when the new app first hits the streets and so it should be relatively simple to grab a private copy in advance of actually needing to use it.
Cheers,
Gary.
RE: Who is the first/next
)
Are you putting up your hand?? :-).
Are we going to add ziegenmelker to the illustrious list of names such as Alex Kan et al, Simon Zadra et al, Jason Gee et al, JDWhale, etc, etc, who have done this sort of thing in the Seti world?? :-).
Actually, Bernd and others behind the scenes (with notable user support from people like Akos and Bikeman) seem to be doing a pretty decent job of tweaking the apps for best performance so perhaps it will be difficult for others to make further significant improvement.
We tend to lose sight of the fact that when S5R3 started at the end of September last year, the initial indications were that the run was going to take well in excess of a year (maybe even 1.5 years) with the apps available at that time. It also took quite a while for significant speed improvements to be made so that the fact that we are now finishing up in about 310 days only is quite amazing. The improved performance of the apps has played a very significant part in this. Of course the recent growth in active hosts certainly helps as well.
One other point of interest. I was still receiving significant S5R2 resend work at the end of October last year, more than a month after S5R3 had started. I guess it's going to be the same for at least the next month this time as well.
Cheers,
Gary.
RE: Actually, Bernd and
)
Probably it is worth mentioning that there will be likely some law of diminishing returns for effort with optimisations. There will be some irreducible computational complexity for a given quality of output/analysis. Choice of algorithm, assembly level tweaking, end-runs on compiler quirks etc are all going to impact - but not with endless runtime reductions.
Of course if major aspects of the problem change then one is looking at a renewed/different scope.
Cheers, Mike.
I have made this letter longer than usual because I lack the time to make it shorter ...
... and my other CPU is a Ryzen 5950X :-) Blaise Pascal
Hi all, If we look back on
)
Hi all,
If we look back on the S5R3 run, the speed of the app increased by a factor of 2 to 3. I also think this will be difficult to repeat in S5R4, unless you make changes on the algorithmic level, which is both difficult and risky.
But speed is not everything. Porting the app to additional platforms is another area where volunteer contributions might be useful & welcome (64 bit code, and the Cell or Sparc CPUs come to mind, as well as different GPUs, for which some work has already been done, tho).
However, that's my personal opinion only. I guess Bernd will have more time to expand on this once S5R4 is done.
CU
Bikeman
RE: RE: Actually, Bernd
)
I asked the same question some time ago and mentioned that people would continue to want SSE3, SSE4, etc, etc, etc... but I think it is fair to say that SSE2 seems to have enough performance advantage to be implemented.
As for the active hosts, I would imagine that the misfortunes over at Cosmology will contribute to higher participation here. I know that I'm going to be reevaluating my contribution there, given all the problems with the servers, the applications, and finally the fact that as viewed from my system, it now pays out credit on the level of LHC, which was already amongst the lowest BOINC-wide.
The military jet type I've always been fascinated by is the A-12 / SR-71. It is well noted that the subsonic performance is terrible, with the plane having to dive to be able to break the sonic barrier. Once supersonic though, nothing (we know of) can match it. If a missle is fired at it, the standard procedure is to just accelerate and/or accelerate and climb...
RE: I asked the same
)
Well, ideally we could have a broad suite of ultra-fine tuned highly optimised app versions, one for each known possible target processor variant on the planet. Maybe that'll be in place by the time we're doing S7R1 and beyond! :-)
I note the scheduled outage this w/end, at a good time b/w runs to drop upgrades in. I wonder what sort of new/hot gear is going in?
Cheers, Mike.
( edit ) Might I also congratulate the project management, developers and all manner of testers and contributors [ and mods :-)] for a pretty seamless run. Their dedication, responsiveness and attention to detail has yielded a very well performing DC project. Touch wood!
I have made this letter longer than usual because I lack the time to make it shorter ...
... and my other CPU is a Ryzen 5950X :-) Blaise Pascal
RE: Well, ideally we could
)
LOL...
Regarding the first remark... Good One! :-D
Pertaining to the second... There goes their continuous uptime record! Still, pretty impressive though. ;-)
Alinator