S5R5 plans

log in

Advanced search

Message boards : Cruncher's Corner : S5R5 plans

1 · 2 · 3 · 4 . . . 5 · Next
Author Message
Filipe
Send message
Joined: 10 Mar 05
Posts: 134
Credit: 45,578,530
RAC: 24,461
Message 90858 - Posted: 30 Oct 2008, 19:06:31 UTC

Any news about the S5R5 run mentioned earlier in this thread?? it was supposed to begin soon, because of some kind of bug on the results of the S5R4...
____________

Profile Bernd Machenschalk
Volunteer moderator
Project administrator
Project developer
Avatar
Send message
Joined: 15 Oct 04
Posts: 3611
Credit: 128,493,759
RAC: 55,194
Message 90864 - Posted: 30 Oct 2008, 21:38:09 UTC - in response to Message 90858.
Last modified: 30 Oct 2008, 21:49:20 UTC

Any news about the S5R5 run mentioned earlier in this thread?? it was supposed to begin soon, because of some kind of bug on the results of the S5R4...

Will definitely come. We're still testing and tweaking the setup. Some facts so far:
- slightly increased memory requirement
- larger "dwell time" per sky location. This is limiting the maximum checkpoint rate to about once per 3 min (on current average CPUs, longer on slower ones)
- workunits will run roughly half as long as S5R4 ones

BM
Profile MarkJ
Avatar
Send message
Joined: 28 Feb 08
Posts: 309
Credit: 34,665,249
RAC: 9,554
Message 90887 - Posted: 31 Oct 2008, 11:55:10 UTC - in response to Message 90864.

Any news about the S5R5 run mentioned earlier in this thread?? it was supposed to begin soon, because of some kind of bug on the results of the S5R4...

Will definitely come. We're still testing and tweaking the setup. Some facts so far:
- slightly increased memory requirement
- larger "dwell time" per sky location. This is limiting the maximum checkpoint rate to about once per 3 min (on current average CPUs, longer on slower ones)
- workunits will run roughly half as long as S5R4 ones

BM


Hi Bernd,

Can we use the current (605) power app, or will we need a new app/app_info?
____________
BOINC blog
Profile Bikeman (Heinz-Bernd Eggenstein)
Volunteer moderator
Project administrator
Project developer
Avatar
Send message
Joined: 28 Aug 06
Posts: 3502
Credit: 149,275,250
RAC: 108,329
Message 90888 - Posted: 31 Oct 2008, 12:07:25 UTC - in response to Message 90887.

Any news about the S5R5 run mentioned earlier in this thread?? it was supposed to begin soon, because of some kind of bug on the results of the S5R4...

Will definitely come. We're still testing and tweaking the setup. Some facts so far:
- slightly increased memory requirement
- larger "dwell time" per sky location. This is limiting the maximum checkpoint rate to about once per 3 min (on current average CPUs, longer on slower ones)
- workunits will run roughly half as long as S5R4 ones

BM


Hi Bernd,

Can we use the current (605) power app, or will we need a new app/app_info?


S5R5 will require new binaries that are currently under test.
CU
Bikeman

____________
John Clark
Avatar
Send message
Joined: 4 May 07
Posts: 1092
Credit: 3,143,193
RAC: 0
Message 90892 - Posted: 31 Oct 2008, 13:28:23 UTC - in response to Message 90888.
Last modified: 31 Oct 2008, 13:31:13 UTC

S5R5 will require new binaries that are currently under test.
CU
Bikeman


I presume there will be a similar rundown of S5R4 and change over to S5R5 as there was for the start of the S5R4? Meaning those that want to initially stick to the S5R4 WUs can do so until these get scarce?

One driver will be the credit, and RAC, given between the R4 and R5. If similar then own choices, if the R5 is slightly better, then the run down of R4 may take longer than planned.
____________
Shih-Tzu are clever, cuddly, playful and rule!! Jack Russell are feisty!

Profile Bernd Machenschalk
Volunteer moderator
Project administrator
Project developer
Avatar
Send message
Joined: 15 Oct 04
Posts: 3611
Credit: 128,493,759
RAC: 55,194
Message 90894 - Posted: 31 Oct 2008, 15:12:31 UTC - in response to Message 90887.
Last modified: 31 Oct 2008, 15:14:06 UTC

Can we use the current (605) power app, or will we need a new app/app_info

The S5R5 Windows App will feature the code that makes the S5R4 6.05 App as fast as it is.

There is no change in crediting targeted for S5R5.

BM
Profile Bikeman (Heinz-Bernd Eggenstein)
Volunteer moderator
Project administrator
Project developer
Avatar
Send message
Joined: 28 Aug 06
Posts: 3502
Credit: 149,275,250
RAC: 108,329
Message 90895 - Posted: 31 Oct 2008, 15:28:16 UTC

Also, the S5R5 run will work on the existing S5R4 datafiles, so the transition form S5R4 to S5R5 should be much smoother than the previous one from S5R3 to S5R4 where it was not possible to re-use downloaded datafiles from the previous run.

CU
Bikeman
____________

John Clark
Avatar
Send message
Joined: 4 May 07
Posts: 1092
Credit: 3,143,193
RAC: 0
Message 90900 - Posted: 31 Oct 2008, 18:13:42 UTC - in response to Message 90895.

Also, the S5R5 run will work on the existing S5R4 datafiles, so the transition form S5R4 to S5R5 should be much smoother than the previous one from S5R3 to S5R4 where it was not possible to re-use downloaded datafiles from the previous run.

CU
Bikeman


That sounds good then!
____________
Shih-Tzu are clever, cuddly, playful and rule!! Jack Russell are feisty!

Benjamin Rietveld
Send message
Joined: 14 May 06
Posts: 43
Credit: 509,320
RAC: 0
Message 90901 - Posted: 31 Oct 2008, 18:18:39 UTC

Partly on topic:

If this run hits, what's the best way to get rid of the optimized application? Resetting the project is what I usually do, but there must be better ways
____________

RandyC
Avatar
Send message
Joined: 18 Jan 05
Posts: 337
Credit: 69,064,654
RAC: 31,898
Message 90911 - Posted: 31 Oct 2008, 23:50:34 UTC - in response to Message 90901.

Partly on topic:

If this run hits, what's the best way to get rid of the optimized application? Resetting the project is what I usually do, but there must be better ways


Drain your queue (set no-new-work)
Report results when it's empty
Shutdown BOINC (make sure it's down all the way)
Remove app_info.xml file
Restart BOINC and enable new work
Profile Yin Gang
Send message
Joined: 23 Feb 05
Posts: 53
Credit: 6,554,084
RAC: 0
Message 90969 - Posted: 2 Nov 2008, 2:10:45 UTC
Last modified: 2 Nov 2008, 2:11:43 UTC

So...the result of S5R4 would still make any sense? If not, I'd like to stop running this project until S5R5 comes out.

YG
____________


Welcome To Team China!

Erik
Send message
Joined: 14 Feb 06
Posts: 2884
Credit: 2,645,600
RAC: 0
Message 90977 - Posted: 2 Nov 2008, 3:55:24 UTC - in response to Message 90969.

So...the result of S5R4 would still make any sense? If not, I'd like to stop running this project until S5R5 comes out.

YG


I believe all of the science runs are valid and useful to some degree. I suggest completing the run.
____________
kashi
Send message
Joined: 6 Mar 07
Posts: 11
Credit: 6,470,129
RAC: 2,477
Message 90981 - Posted: 2 Nov 2008, 8:19:56 UTC

Well done, that's excellent that the new application will have shorter running tasks. I hope you reduce the deadline accordingly. I would suggest 7 days instead of the current 18 days would help greatly to reduce the length of time that some tasks remain pending.

7 days would be long enough for even the slowest computers to complete the new shorter tasks. I know there are some who still use museum pieces for fun but there comes a time when they are no longer suited for distributed computing for electricity/carbon reasons.

A shorter deadline also helps to minimise total pending time when tasks remain unsent for 7-10 days.

It also minimises the delay caused by those who download a large number of tasks, only complete a few and then detach and reattach and repeat this procedure.

[B^S] Elphidieus
Send message
Joined: 20 Feb 05
Posts: 220
Credit: 12,347,408
RAC: 0
Message 90990 - Posted: 2 Nov 2008, 12:43:38 UTC - in response to Message 90981.

Well done, that's excellent that the new application will have shorter running tasks. I hope you reduce the deadline accordingly. I would suggest 7 days instead of the current 18 days would help greatly to reduce the length of time that some tasks remain pending.

7 days would be long enough for even the slowest computers to complete the new shorter tasks. I know there are some who still use museum pieces for fun but there comes a time when they are no longer suited for distributed computing for electricity/carbon reasons.

A shorter deadline also helps to minimise total pending time when tasks remain unsent for 7-10 days.

It also minimises the delay caused by those who download a large number of tasks, only complete a few and then detach and reattach and repeat this procedure.


A seven-day deadline is way too short as it will be detrimental (pardon my strong word) to those multi-core crunchers like me who would download loads of workunits to last a week or two on less-accessible-yet-automated machines.

Can't wait to get credited on your work...?
kashi
Send message
Joined: 6 Mar 07
Posts: 11
Credit: 6,470,129
RAC: 2,477
Message 90992 - Posted: 2 Nov 2008, 13:28:58 UTC
Last modified: 2 Nov 2008, 14:12:22 UTC

I don't mind waiting a few weeks but a month or more is excessive on any project.

I understand that many who download a lot of work at once complete it all but there are also others with hidden computers who make a habit of downloading many tasks, wait until their wingmen have completed them and then only complete the faster tasks. Not only is this against the spirit of crunching but it is wasteful of server resources and bandwidth.

Brian Silvers
Send message
Joined: 26 Aug 05
Posts: 782
Credit: 282,700
RAC: 0
Message 91002 - Posted: 2 Nov 2008, 17:43:48 UTC - in response to Message 90992.

I don't mind waiting a few weeks but a month or more is excessive on any project.

I understand that many who download a lot of work at once complete it all but there are also others with hidden computers who make a habit of downloading many tasks, wait until their wingmen have completed them and then only complete the faster tasks. Not only is this against the spirit of crunching but it is wasteful of server resources and bandwidth.


The way the datasets are distributed, if you reduce to a 7-day deadline you will more than likely significantly increase the amount of "backfill" downloading that goes on. This means that there will be more downloading of 70MB+ groups of files. This will irritate those still on dialup.

If runtimes are indeed halved, then the minimum deadline should go to 9 days, since that is half of the current 18. My suggestion though would be to return to the original 14-day deadline.
____________
Profile Bikeman (Heinz-Bernd Eggenstein)
Volunteer moderator
Project administrator
Project developer
Avatar
Send message
Joined: 28 Aug 06
Posts: 3502
Credit: 149,275,250
RAC: 108,329
Message 91018 - Posted: 2 Nov 2008, 21:47:30 UTC

Several weeks ago the "pending credit" situation was rather bad as reported in this thread, but now, at least for me, it's OK again. If it stays like it is now I can live with a 18 or 14 day deadline. I guess the fact that ATLAS stopped crunching for some time when the servers got overloaded must have contributed to the massive increase of pending credits ?

CU
Bikeman

____________

archae86
Send message
Joined: 6 Dec 05
Posts: 1761
Credit: 355,840,764
RAC: 570,850
Message 91030 - Posted: 2 Nov 2008, 23:19:23 UTC - in response to Message 91018.

I guess the fact that ATLAS stopped crunching for some time when the servers got overloaded must have contributed to the massive increase of pending credits ?
Not directly by waiting for it, I think. It seems to me the primary symptom was that it became common for many days to go by between first issue of a result from a WU and issue of the first quorum partner result in that same WU. No matter how promptly everyone processes the results they receive, that situation gives trouble.

It seems to have become less common recently, though my tiny fleet is not a big enough sample to say that with any assurance. Even in that fleet I spotted some 5 day delays within the last week.

Perhaps ATLAS contributed indirectly: instead of waiting for it our waits were a consequence of the scheduler's poor response to its presence.

____________
Brian Silvers
Send message
Joined: 26 Aug 05
Posts: 782
Credit: 282,700
RAC: 0
Message 91033 - Posted: 2 Nov 2008, 23:28:36 UTC - in response to Message 91030.

No matter how promptly everyone processes the results they receive, that situation gives trouble.


Could you please state the "trouble" it gives? Help me understand why there is a problem...
____________
Profile Bernd Machenschalk
Volunteer moderator
Project administrator
Project developer
Avatar
Send message
Joined: 15 Oct 04
Posts: 3611
Credit: 128,493,759
RAC: 55,194
Message 91048 - Posted: 3 Nov 2008, 9:59:46 UTC
Last modified: 3 Nov 2008, 10:50:36 UTC

The "deadline" is something that can rather easily be adjusted on-the-fly during a run (whereas the average workunit duration is not). With current workunits the average is set to be 18d, for what I recall from previous discussions I think most people feel comfortable with about 14d for 6-8h WUs.

BM

1 · 2 · 3 · 4 . . . 5 · Next

Message boards : Cruncher's Corner : S5R5 plans


Home · Your account · Message boards

This material is based upon work supported by the National Science Foundation (NSF) under Grants PHY-1104902, PHY-1104617 and PHY-1105572 and by the Max Planck Gesellschaft (MPG). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the investigators and do not necessarily reflect the views of the NSF or the MPG.

Copyright © 2016 Bruce Allen