I'd rather to have that on the frontpage in the news. Really. It's not much used anyway, example: Last mention of S4 was on 6/16, but no mention when a S4-report will be available.
We're working hard on the S4 postprocessing, but I can't predict when it will be finished. One of the characteristics of research work is that it is open-ended, and that makes it less deterministic than many other kinds of work.
I'd rather to have that on the frontpage in the news. Really. It's not much used anyway, example: Last mention of S4 was on 6/16, but no mention when a S4-report will be available.
We're working hard on the S4 postprocessing, but I can't predict when it will be finished. One of the characteristics of research work is that it is open-ended, and that makes it less deterministic than many other kinds of work.
He, thanks for the input; that would make some good front-news, you know :)
Cheers, Alex.
"I am tired of all this sort of thing called science here... We have spent
millions in that sort of thing for the last few years, and it is time it
should be stopped."
-- Simon Cameron, U.S. Senator, on the Smithsonian Institute, 1901.
Yes, they are, which is the whole point about my posting, a very important effect indeed. If you have a look over at the Rosetta@Home project, there's currently a flamewar going on, a really really bad one - no need to read it in full, ugly length; one participant (to keep consistent with your terminology), a Mac PPC user, found out that the Rosetta-application utilized the CPU completly but used only 20%-30% for real work - he was totally pissed off and angry.
This is one of those communication-GAUs(*) i was hinting too. Your recent credit-calibration fauxpas i was reffering too is nothing compared to that one.
Quote:
I've been following the flamewar at Rosetta (I have my slowest computer attached to that project). The root of the whole thing seems to be an argument over credits per unit time. The official Rosetta applications are not optimized to use SSE+. Somebody came up with an unofficial optimized app, which allowed some participants to earn more credit per unit time than others. Some of the "others" complained about cheating on the part of those using the optimized app (mainly the top team) and Rosetta adjusted the credit granting process to take that speed advantage away. That was (as far as I can tell) the start of the Rosetta flamewar and ultimately resulted in the top team detaching from the project and taking their 2,000+ computers elsewhere. Everything else has been an argument over adopting optimized apps versus continuing the use of non-optimized apps.
Goes to show what happens when credits become more important than the science and when emotions are allowed to overrule reason. The end result of the Rosetta war will be a delay in the science--which actually means something--because of meaningless credits.
I'd rather to have that on the frontpage in the news. Really. It's not much used anyway, example: Last mention of S4 was on 6/16, but no mention when a S4-report will be available.
We're working hard on the S4 postprocessing, but I can't predict when it will be finished. One of the characteristics of research work is that it is open-ended, and that makes it less deterministic than many other kinds of work.
LOL, not to mention that when you start to think you've got a handle on all the answers, the questions all get changed!
Goes to show what happens when credits become more important than the science and when emotions are allowed to overrule reason. The end result of the Rosetta war will be a delay in the science--which actually means something--because of meaningless credits.
Thought I'd better clarify my statements before a flamewar starts over here. I think that both sides of the Rosetta war have completely forgotten about the science. They have also forgotten that while credits are nice, those credits are ultimately meaningless in the real world. Therefore, instead of wanting an optimized app to help speed the project, some crunchers complained that optimized apps gave other crunchers an unfair advantage in the credit system. And the team that left decided to leave partly because they would not have their credit-earning advantage given by their optimized application (being called cheaters didn't help retain them, either). The end results are that Rosetta isn't able to compute as much, retarding the progress of the project and a lot of bad blood between participants. In my view, it's a tempest in a teacup that is resulting in bad consequences for all involved.
Just to show you that I am not immune to the psycological aspects of earning credits, I convinced my dissertation advisor to allow me to attach the laboratory computer to Einstein in an attempt to keep my RAC up when the Einstein credits/work unit dropped. So in a way, I'm no different from the warmongers in Rosetta--psycologically I respond to credits. I'd just like to get my hands on that optimized app rather than complain about its use.
[...] That was (as far as I can tell) the start of the Rosetta flamewar and ultimately resulted in the top team detaching from the project and taking their 2,000+ computers elsewhere. Everything else has been an argument over adopting optimized apps versus continuing the use of non-optimized apps.
Goes to show what happens when credits become more important than the science and when emotions are allowed to overrule reason. The end result of the Rosetta war will be a delay in the science--which actually means something--because of meaningless credits.
Thanks for the information... I didn't dig that deep.
Alex.
"I am tired of all this sort of thing called science here... We have spent
millions in that sort of thing for the last few years, and it is time it
should be stopped."
-- Simon Cameron, U.S. Senator, on the Smithsonian Institute, 1901.
Thought I'd better clarify my statements before a flamewar starts over here.
I understood this thread more to be about the psychology/sociology of DC and not about the pros and cons of a certain credit-system in this or any other project.
Alex.
"I am tired of all this sort of thing called science here... We have spent
millions in that sort of thing for the last few years, and it is time it
should be stopped."
-- Simon Cameron, U.S. Senator, on the Smithsonian Institute, 1901.
I've been following the flamewar at Rosetta (I have my slowest computer attached to that project). The root of the whole thing seems to be an argument over credits per unit time. The official Rosetta applications are not optimized to use SSE+. Somebody came up with an unofficial optimized app, which allowed some participants to earn more credit per unit time than others. Some of the "others" complained about cheating on the part of those using the optimized app (mainly the top team) and Rosetta adjusted the credit granting process to take that speed advantage away. That was (as far as I can tell) the start of the Rosetta flamewar and ultimately resulted in the top team detaching from the project and taking their 2,000+ computers elsewhere. Everything else has been an argument over adopting optimized apps versus continuing the use of non-optimized apps.
Goes to show what happens when credits become more important than the science and when emotions are allowed to overrule reason. The end result of the Rosetta war will be a delay in the science--which actually means something--because of meaningless credits.
My two cents.
Jim Milks
I'd like to clarify on the credit war at Rosetta. There is and never was an optimized application for Rosetta. Optimising the Rosetta app turns out to be very difficult due to the nature of the algorithm. The project team and others try optimizing the app all the time but so far there is no use of SSE or AMD64, so the fastest and only app is still the official one.
What happened was that people used the optimized boinc client as provided by Truxoft and Crunch3r. Due to the quorum of 1 on Rosetta those hosts got what they claimed although they did not compute more work for Rosetta than hosts which use the standard boinc client (I hope by now everybody knows that the optimized boinc client just claims more credit but does not speed up the science application). This led to very unequal credit granting for the same amount of work and it was long ago decided that Rosetta will switch to a work based credit system after CASP was over. In the transition to the new credit system some people accused XtremeSystems that they were cheating all the time by using the boinc version 5.5.0 from Crunch3er (btw a positions which is shared by Crunch3r himself). The project team deleted those accusations but due to a moderator shortage at this time it took some time. Flaming continued for a while although the project stated that they do not consider this as cheating and are grateful to any contribution they got but that they however think the new credit system will be much fairer and put those accusations on hold. XS for some reason was not satisfied with these comments and demanded a special post from the project leader in which he should blame certain users and defend the position from XS. That demand was not fullfilled so they pulled out. Btw they had never a combined force of 2000+ hosts, more likely about 1000.
This post is just meant to clarify what happened at Rosetta.
Well, UCB conducted a survey which I also quote in parts; I also read elsewhere that you basically found that survey kind of useless because the questionary itself was not meaningful enough - from a scientific point of view.
I can only imagine how a scientifically good survey would look like. I just know the "employee satisfaction surveys" my company make every thre months: The question are all kind of strange and awkward, all the time - i suspect that there's a system behind it which tells those people more about the "collective satisfaction" than i possibly could imagine.
So, would you say that conducting a scientifically constructed (but maybe awkward) survey would push the understanding of what the participants (let's stick to this term) want/adore/dislike ahead? If yes, is it possible to create such a survey on our own according to "scheme F" or would you suggest to hire people for this?
Yes, my thoughts on the UCB survey are that it looks like it was created by a high school social science class at best (and I am probably underestimating the high schoolers by doing so). For example, the demographic questions on the survey are poorly constructed including age categories such as 20-39. Such categories are clearly worthless (just how useful is it to lump 20 year-old college students with their 39 year-old professors?). I'd also add that, though I have not seen the "employee satisfaction" surveys from your company, in my experience most are done badly--though not at a level as poor as that done by UCB.
To answer your specific questions: Yes, a scientifically constructed (and implemented--we can't forget issues of proper sampling) survey will yield a greater understanding of participant motivations as well as potentially considerable information on how DC is actually working (and it need not be "awkward" in how it reads).
As for the participant base constructing the survey, the skill set to do so is almost certainly here (I, for example, am a professor who teaches classes in social science research methods and statistics to PhD students--though I am much to busy these days with my own work to contribute much to such an endeavor). However, it is usually not a good idea to have the people you are surveying also be the one's conducting and creating the survey. Hiring someone is almost certainly needed, though the costs are likely not extraordinary. For example, one could fund an advanced social science graduate student for one year with $20,000 (and UCB certainly should have a significant pool of such students given its excellent programs in sociology, demography, psychology, and political science). Implementation costs of the survey could probably be kept very minimal since a web-based design would be sufficient given the 'on-line' nature of participants. For about twice the grad student costs, one could alternatively hire a social science post-doctoral fellow to do this (salary would typically be $35,000-40,000 with benefits).
Personally, I'd settle for one point per valid WU completed.
Probably a lot of people would abort the long ones.
I totally agree with Bert. That's what people would do.
Quote:
I am curious, what you mean with to "be as fair as possible with the credits", Alexander...
First of all i want to point out that i din't point on any project, caliming "OMG! Their credit-system is totally fscked up!". The fact that the projects Einstein and Rosetta appeared in my article was that I'm reading the fori of those projects and that I'm participating. My rant was more about communication, not about the fairness of a certain credit-system. However, i opened the can of worms so i have to show my colours.
Actually, your article looks like it's about credit systems, not about communication...
Quote:
My personal opinion is about "scientific work done" - not about FLOPS or MIPS. MIPS and FLOPS are a fixed value of your CPU and OS but more effective algorithms can produce more results per instructions. If you got an algorithm running in O(n) and one running O(log(n)) there's no discussion about which algorithm will do more work per FLOPS or MIPS.
So using MIPS and FLOPS is *no* benchmark at all IMHO.
Agreed.
Quote:
Participants should be rewarded of "contributed scientific work".
Agreed again.
Quote:
But i also got one point: Lot's of people are dedicating their oldish machines to projects - the only very existance of those "vintage machines" is to crunch work-units for the project. Slow, but steady. That's something which really should be rewarded, it's a sign of DEDICATION.
That's just my personal opinion.
Unfortunately, it's about the work done. Or more specific: on the efficiency. I also applaud that people dedicate their machines to projects like Einstein. But I don't agree it should be at ALL cost. Old machines are at the end of their economic lifes, and take more energy than a newer machine. A bit more is explained in this thread. An ideal crediting system would take such a ratio in account.
Quote:
Quote:
[...] I'd say, work on a continuous calibration, to keep average credit level for every day for all projects, instead only after an improvement of the apps for a single project.
Sounds like a request for a standard-workunit? Even if i got you wrong, it still sounds like a good idea though...
Cheers, Alex.
A standard WU? At the moment I can't see such a thing happening. I only run Einstein, but what I can tell from stories, WUs from other project require different sizes of WUs, as well as other algorithms. I did not specifically look into it, but if I've got it right, Einstein uses Fourier transforms, just as Seti does. I can imagine that CPDN would not use this. So this would make a standard WU not feasible.
I'd say: collect all results for one day. Instead of claiming credit, a result should claim weight (how much work it relatively took to process). The ratio of required average credit factored with the number of results and the sum of weights of the results gives a correction factor. Multiplication of this correction factor with the granted weight for each WU would then give credit. This way, the average granted credit would be constant per day. No sudden changes in the crediting system would be needed to keep a "fair" inter-project crediting system.
Main disadvantage would be that a WU could be worth less credits on another day. For the competition inside the project, also a ranking of the weights should be maintained.
RE: I'd rather to have that
)
We're working hard on the S4 postprocessing, but I can't predict when it will be finished. One of the characteristics of research work is that it is open-ended, and that makes it less deterministic than many other kinds of work.
Director, Einstein@Home
RE: RE: I'd rather to
)
He, thanks for the input; that would make some good front-news, you know :)
Cheers, Alex.
"I am tired of all this sort of thing called science here... We have spent
millions in that sort of thing for the last few years, and it is time it
should be stopped."
-- Simon Cameron, U.S. Senator, on the Smithsonian Institute, 1901.
RE: Yes, they are, which is
)
RE: RE: I'd rather to
)
LOL, not to mention that when you start to think you've got a handle on all the answers, the questions all get changed!
Alinator
RE: Goes to show what
)
Thought I'd better clarify my statements before a flamewar starts over here. I think that both sides of the Rosetta war have completely forgotten about the science. They have also forgotten that while credits are nice, those credits are ultimately meaningless in the real world. Therefore, instead of wanting an optimized app to help speed the project, some crunchers complained that optimized apps gave other crunchers an unfair advantage in the credit system. And the team that left decided to leave partly because they would not have their credit-earning advantage given by their optimized application (being called cheaters didn't help retain them, either). The end results are that Rosetta isn't able to compute as much, retarding the progress of the project and a lot of bad blood between participants. In my view, it's a tempest in a teacup that is resulting in bad consequences for all involved.
Just to show you that I am not immune to the psycological aspects of earning credits, I convinced my dissertation advisor to allow me to attach the laboratory computer to Einstein in an attempt to keep my RAC up when the Einstein credits/work unit dropped. So in a way, I'm no different from the warmongers in Rosetta--psycologically I respond to credits. I'd just like to get my hands on that optimized app rather than complain about its use.
Jim Milks
RE: [...] That was (as far
)
Wooahh. Woah. Scary. That's really something. Disturbing.
Thanks for the information... I didn't dig that deep.
Alex.
"I am tired of all this sort of thing called science here... We have spent
millions in that sort of thing for the last few years, and it is time it
should be stopped."
-- Simon Cameron, U.S. Senator, on the Smithsonian Institute, 1901.
RE: Thought I'd better
)
I understood this thread more to be about the psychology/sociology of DC and not about the pros and cons of a certain credit-system in this or any other project.
Alex.
"I am tired of all this sort of thing called science here... We have spent
millions in that sort of thing for the last few years, and it is time it
should be stopped."
-- Simon Cameron, U.S. Senator, on the Smithsonian Institute, 1901.
RE: I've been following
)
I'd like to clarify on the credit war at Rosetta. There is and never was an optimized application for Rosetta. Optimising the Rosetta app turns out to be very difficult due to the nature of the algorithm. The project team and others try optimizing the app all the time but so far there is no use of SSE or AMD64, so the fastest and only app is still the official one.
What happened was that people used the optimized boinc client as provided by Truxoft and Crunch3r. Due to the quorum of 1 on Rosetta those hosts got what they claimed although they did not compute more work for Rosetta than hosts which use the standard boinc client (I hope by now everybody knows that the optimized boinc client just claims more credit but does not speed up the science application). This led to very unequal credit granting for the same amount of work and it was long ago decided that Rosetta will switch to a work based credit system after CASP was over. In the transition to the new credit system some people accused XtremeSystems that they were cheating all the time by using the boinc version 5.5.0 from Crunch3er (btw a positions which is shared by Crunch3r himself). The project team deleted those accusations but due to a moderator shortage at this time it took some time. Flaming continued for a while although the project stated that they do not consider this as cheating and are grateful to any contribution they got but that they however think the new credit system will be much fairer and put those accusations on hold. XS for some reason was not satisfied with these comments and demanded a special post from the project leader in which he should blame certain users and defend the position from XS. That demand was not fullfilled so they pulled out. Btw they had never a combined force of 2000+ hosts, more likely about 1000.
This post is just meant to clarify what happened at Rosetta.
RE: Well, UCB conducted a
)
Yes, my thoughts on the UCB survey are that it looks like it was created by a high school social science class at best (and I am probably underestimating the high schoolers by doing so). For example, the demographic questions on the survey are poorly constructed including age categories such as 20-39. Such categories are clearly worthless (just how useful is it to lump 20 year-old college students with their 39 year-old professors?). I'd also add that, though I have not seen the "employee satisfaction" surveys from your company, in my experience most are done badly--though not at a level as poor as that done by UCB.
To answer your specific questions: Yes, a scientifically constructed (and implemented--we can't forget issues of proper sampling) survey will yield a greater understanding of participant motivations as well as potentially considerable information on how DC is actually working (and it need not be "awkward" in how it reads).
As for the participant base constructing the survey, the skill set to do so is almost certainly here (I, for example, am a professor who teaches classes in social science research methods and statistics to PhD students--though I am much to busy these days with my own work to contribute much to such an endeavor). However, it is usually not a good idea to have the people you are surveying also be the one's conducting and creating the survey. Hiring someone is almost certainly needed, though the costs are likely not extraordinary. For example, one could fund an advanced social science graduate student for one year with $20,000 (and UCB certainly should have a significant pool of such students given its excellent programs in sociology, demography, psychology, and political science). Implementation costs of the survey could probably be kept very minimal since a web-based design would be sufficient given the 'on-line' nature of participants. For about twice the grad student costs, one could alternatively hire a social science post-doctoral fellow to do this (salary would typically be $35,000-40,000 with benefits).
Hi Alexander! RE: Bert
)
Hi Alexander!
Actually, your article looks like it's about credit systems, not about communication...
Agreed.
Agreed again.
Unfortunately, it's about the work done. Or more specific: on the efficiency. I also applaud that people dedicate their machines to projects like Einstein. But I don't agree it should be at ALL cost. Old machines are at the end of their economic lifes, and take more energy than a newer machine. A bit more is explained in this thread. An ideal crediting system would take such a ratio in account.
A standard WU? At the moment I can't see such a thing happening. I only run Einstein, but what I can tell from stories, WUs from other project require different sizes of WUs, as well as other algorithms. I did not specifically look into it, but if I've got it right, Einstein uses Fourier transforms, just as Seti does. I can imagine that CPDN would not use this. So this would make a standard WU not feasible.
I'd say: collect all results for one day. Instead of claiming credit, a result should claim weight (how much work it relatively took to process). The ratio of required average credit factored with the number of results and the sum of weights of the results gives a correction factor. Multiplication of this correction factor with the granted weight for each WU would then give credit. This way, the average granted credit would be constant per day. No sudden changes in the crediting system would be needed to keep a "fair" inter-project crediting system.
Main disadvantage would be that a WU could be worth less credits on another day. For the competition inside the project, also a ranking of the weights should be maintained.
Regards,
Bert
Somnio ergo sum