Gravitational Wave S6 LineVeto search (extended) ending

Einstein@Home is slowly but steadily finishing the current Gravitational Wave search. The data used for this search will not be re-used for subsequent searches. This means that our "locality scheduling" won't be as effective as before to keep the download volume low. In this finishing phase you may receive tasks that require you to download 100MB or more to process. If you don't want this to happen, you may suspend the Einstein@Home project for a week or opt-out of that application in your EInstein@Home preferences (on "your account" page).

BM

Comments

Ryan
Ryan
Joined: 3 Oct 12
Posts: 4
Credit: 1573966
RAC: 0

Gravitational Wave S6 LineVeto search (extended) ending

Thanks for that, I guess my computer will be quite tough to calculate those files. The smallest one which is 14000 GFLOPS requires me 2 hours and half to finish. That one probably 7 hours each

Go Einstein@home.

Nigel Garvey
Nigel Garvey
Joined: 4 Oct 10
Posts: 50
Credit: 16823285
RAC: 77865

RE: Einstein@Home is slowly

Quote:
Einstein@Home is slowly but steadily finishing the current Gravitational Wave search

Will there (eventually) be anything to replace it for PPC Macs?

NG

Betreger
Betreger
Joined: 25 Feb 05
Posts: 987
Credit: 1421487409
RAC: 811310

That's too bad, since my main

That's too bad, since my main interest in this project is the Gravitational Wave search. I have enjoyed the professional way this project is run but without gravity waves to search for my interest wanes. I crunch BRPs but I do not understand how they relate to the gravity wave search. I do understand we have run out of data so when it is gone I will look to see if I can find another project to support. When the LIGO observatories are back up and hopefully supplying new data I know I will be back.

Bernd Machenschalk
Bernd Machenschalk
Moderator
Administrator
Joined: 15 Oct 04
Posts: 4265
Credit: 244921706
RAC: 16859

This doesn't mean that

This doesn't mean that Einstein@Home is abandoning the search for Gravitational Waves altogether. It just means that the current "run" that analyzes this particular set of data is coming to an end. It will be superseded by another search for Gravitational Waves, which is planned to be started next week (and already being tested on Albert@Home). This uses essentially the same raw data (from the sixth science run S6), but with slightly different pre-processing, thus new data files will need to be distributed to the Clients.

BM

BM

Betreger
Betreger
Joined: 25 Feb 05
Posts: 987
Credit: 1421487409
RAC: 811310

Thank you, this spares me

Thank you, this spares me from searching for a new project.

Aaron Dodds
Aaron Dodds
Joined: 1 Feb 13
Posts: 8
Credit: 2102404
RAC: 0

So how come if we are using

So how come if we are using the same raw data are we going to find any other gravitational waves ?

Mike Hewson
Mike Hewson
Moderator
Joined: 1 Dec 05
Posts: 6534
Credit: 284700169
RAC: 113726

RE: So how come if we are

Quote:
So how come if we are using the same raw data are we going to find any other gravitational waves ?


Basically the 'sound' of a wave can vary - and no one has heard one yet - so different templates ( signal shapes ) and search methods are being tried. Such permutations have yet to be exhausted. Generally the noise is far greater than the signal of interest and so there is extra subtlety due to that. For that matter, no waves may ever be heard and that is one point of such investigations : to determine whether General Relativity is correct in that regard.

Cheers, Mike.

I have made this letter longer than usual because I lack the time to make it shorter ...

... and my other CPU is a Ryzen 5950X :-) Blaise Pascal

Jeroen
Jeroen
Joined: 25 Nov 05
Posts: 379
Credit: 740030628
RAC: 556

Thanks for the update. I am

Thanks for the update. I am looking forward to the next search for gravitational waves.

Betreger
Betreger
Joined: 25 Feb 05
Posts: 987
Credit: 1421487409
RAC: 811310

Is this project using noise

Is this project using noise cancelling technology in the signal processing.

Mike Hewson
Mike Hewson
Moderator
Joined: 1 Dec 05
Posts: 6534
Credit: 284700169
RAC: 113726

RE: Is this project using

Quote:
Is this project using noise cancelling technology in the signal processing.


In the sense that known sources are accounted for like 60Hz and harmonics. IIRC essentially white/Gaussian noise is substituted at some suitable level, but I can't remember exactly where in the pipeline that is done. But there are many more under the heading of 'noise budget'. This is where a good understanding of the mechanics of the interferometer operations comes into play ( it's a transducer of spacetime strain to photon counts ).

[ So I guess that if an actual 60Hz gravitational wave came in, say, we'd miss it on account of randomising that channel. But sky sources are quite a distance away so an exact 60Hz signal right now today is going to Doppler away from that, if only due to Earth's motion. ]

NB : All computation techniques must cease in a finite time and with that comes two risks : firstly that a wave will missed ( some threshold too high ), and secondly that too many false alarms are generated ( some threshold too low ).

Cheers, Mike.

I have made this letter longer than usual because I lack the time to make it shorter ...

... and my other CPU is a Ryzen 5950X :-) Blaise Pascal

Betreger
Betreger
Joined: 25 Feb 05
Posts: 987
Credit: 1421487409
RAC: 811310

Mike, thanx for the insight.

Mike, thanx for the insight.

Bernd Machenschalk
Bernd Machenschalk
Moderator
Administrator
Joined: 15 Oct 04
Posts: 4265
Credit: 244921706
RAC: 16859

RE: IIRC essentially

Quote:
IIRC essentially white/Gaussian noise is substituted at some suitable level, but I can't remember exactly where in the pipeline that is done.

This is actually the main difference between the old and the new set of data: The old one has been "cleaned" more extensively in the pre-processing (e.g. known instrumental lines replaced by white noise). The new data is almost raw. Essentially the new "Line Veto" statistics (a misnomer actually, because it's not really a veto) is not as sensitive to distractions as previous techniques and allows us to better use the extra power / information even from noisier bands that were previously removed.

I thought I wrote about the new search (actually a set of searches) somewhere else about half a year ago, but I can't find it right now. In previous searches we were broadly scanning the whole sky. This time we will focus our senses in a few directions, down to single points. This will allow us to reach deeper into distance than we ever did before. Our first such target will be Cassiopeia A, that's why that run will be named "S6CasA" (as it is already on Albert).

BM

BM

Bernd Machenschalk
Bernd Machenschalk
Moderator
Administrator
Joined: 15 Oct 04
Posts: 4265
Credit: 244921706
RAC: 16859

RE: I thought I wrote about

Quote:
I thought I wrote about the new search (actually a set of searches) somewhere else about half a year ago

There it is. Not very much, though.

BM

BM

Mike Hewson
Mike Hewson
Moderator
Joined: 1 Dec 05
Posts: 6534
Credit: 284700169
RAC: 113726

RE: This will allow us to

Quote:
This will allow us to reach deeper into distance than we ever did before.


Ah, so fixing the two sky angles ( fairly closely at least ) gives us extra ( coherent? ) convolution time per template to play with, so to speak? For a given computation load then better sensitivity for a chosen sky position. Hence yielding either increased distance for a given putative signal strength or increased signal detection chance for a given distance? :-)

Quote:
Our first such target will be Cassiopeia A, that's why that run will be named "S6CasA" (as it is already on Albert).


Hmmmm, it would be nice if there was some known* EM rhythmic behaviour with such a source to cue/hint the frequency search choices.

Cheers, Mike.

* But then again it would be real cool if the first GW ever found was characterised prior to an EM detection/confirmation.

I have made this letter longer than usual because I lack the time to make it shorter ...

... and my other CPU is a Ryzen 5950X :-) Blaise Pascal

Bikeman (Heinz-Bernd Eggenstein)
Bikeman (Heinz-...
Moderator
Joined: 28 Aug 06
Posts: 3522
Credit: 686042413
RAC: 597943

RE: Hmmmm, it would be

Quote:

Hmmmm, it would be nice if there was some known* EM rhythmic behaviour with such a source to cue/hint the frequency search choices.

Yes, this would make a "targeted" search possible, but then again, without the need to search a rather wide frequency range, such a search could just as well be done on fewer machines in a reasonable time, no need to throw Einstein@Home on it.

Cheers
HB

Bernd Machenschalk
Bernd Machenschalk
Moderator
Administrator
Joined: 15 Oct 04
Posts: 4265
Credit: 244921706
RAC: 16859

RE: such a search could

Quote:
such a search could just as well be done on fewer machines in a reasonable time, no need to throw Einstein@Home on it.

Actually the data-volume / computing-time ratio of this search would make it a pretty bad idea to run it on E@H. Clients would spend more time for downloading the data than actually computing.

BM

BM

IslanderUK
IslanderUK
Joined: 25 Jul 12
Posts: 8
Credit: 2365136
RAC: 0

Hi I'm sorry but I pretty

Hi

I'm sorry but I pretty ignorant about these things even though I am running E@H.

I have read this thread and understood about 20% of it.

I joined this project because I liked the idea of searching for something in space. Has this search come to an end ?

My question is simply, is it worth me sticking with it and letting my computer continue to run E@H or should I go an find some other project.

Holmis
Joined: 4 Jan 05
Posts: 1118
Credit: 1055935564
RAC: 0

RE: Hi I'm sorry but I

Quote:

Hi

I'm sorry but I pretty ignorant about these things even though I am running E@H.

I have read this thread and understood about 20% of it.

I joined this project because I liked the idea of searching for something in space. Has this search come to an end ?

My question is simply, is it worth me sticking with it and letting my computer continue to run E@H or should I go an find some other project.

To keep it really simple, one search for something in space is ending and soon another search for something in space will take it's place.
So just continue to run E@H to search for something in space! =)
And yes it is worth it!

Jonatan
Jonatan
Joined: 20 Jun 10
Posts: 66
Credit: 25782906
RAC: 0

I am happy to participate in

I am happy to participate in the end of the S6 Gravitational Wave search...
We are ending the begin of the future discoveries of Gravitational Waves, and I am also glad to participate in the others series of data

And what happen with the BRP4 series, that now, the time for crunching of one BRP4 has increased 7 times, the normal time of crunching??

My GPUs are sweating :)), but they don´t stop before nothing

Mike Hewson
Mike Hewson
Moderator
Joined: 1 Dec 05
Posts: 6534
Credit: 284700169
RAC: 113726

RE: My question is simply,

Quote:
My question is simply, is it worth me sticking with it and letting my computer continue to run E@H or should I go an find some other project.


My apologies if my/our discussion has caused confusion. For this project one type of search is ending, but new ones are beginning. We were chatting about the pros and cons of computational approaches to examining objects in the sky like Cassiopeia A. This is the first of a series of new searches looking at specific points in the sky, as distinct from the broad area searches we have done before. Being a new thing I think this is quite exciting, but then again I'm well known for my ( excess of ? ) enthusiasm for E@H ! :-) :-)

Cassiopeia A is a curious thing, the remnant of a supernova - an exploding star. That occurred about 300 years ago and has been the subject of much debate by astronomers since. The basic issue is that the thing which exploded was surrounded by alot of other material before it did blow up, and thus it was not clearly seen at the time of the explosion. In my reading it seems there is still doubt ( of course that depends upon which source you are reading and when that was written ) about the exact nature of what was there beforehand, the mechanism of the supernova ( not all stars explode in the same way ) and what remains in that system now.

Bernd and HB correctly pointed out to me that if more was known from standard astronomical techniques then there would not be much point involving E@H. It would be too trivial a search to waste the valuable resource that E@H is. So the difficulty of the Cassiopeia A problem is thus to be matched against the computational grunt of E@H !! :-)

Generally : I sense that some contributors may feel disappointed or downcast when the uncertainty of the process of discovery is mentioned. This field of gravitational wave research is cutting/bleeding edge and has much promise in revealing really new stuff. It is definitely not like my local police sergeant, who surely* knows that every time he goes out to the main highway with his radar gun he will come back with a bag of infringement notices. The gravitational wave detectors are truly new devices and still being developed. Likewise the 'computational backend' that E@H is part of is also new and subject to development.

So as they say for our local Lotto : "you've got to be in it to win it !" :-)

Cheers, Mike.

* Alas, Victorian drivers are that predictable.

I have made this letter longer than usual because I lack the time to make it shorter ...

... and my other CPU is a Ryzen 5950X :-) Blaise Pascal

IslanderUK
IslanderUK
Joined: 25 Jul 12
Posts: 8
Credit: 2365136
RAC: 0

Thank you for you

Thank you for you explanation. I shall stick with it specially as I feel it is more interesting than simply search for extra-terrestrials on Seti

Bernd Machenschalk
Bernd Machenschalk
Moderator
Administrator
Joined: 15 Oct 04
Posts: 4265
Credit: 244921706
RAC: 16859

Just a quick heads-up: The

Just a quick heads-up:

The MC studies that we ran in the background to verify the validity of the "CasA" search showed a significant problem in the application, i.e. for certain parameters it doesn't give the results that we expect. We are still in the process of tracking this down. This will certainly delay the start of the new GW search by a couple of days (or, in worst case, weeks). The original plan was to start the new GW search before we run out of work for the old, which we have been doing for years. Depending on how long it takes to track down and fix this problem, it may actually happen (for the first time IIRC) that Einstein@Home will have no GW "work" for some brief period of time.

For this period, the CPUs attached to this project will be fed work for our other searches, which, however, might get a little tight, too:

- We are running out of data for the BRP4 search. New data from Arecibo has been announced to us, but is currently stuck in Cornell due to technical problems, and we didn't get any yet. We sure hope it will arrive on time, but there's not much we can do about that.

- Newer CPUs process the FGRP workunits so fast that our infrastructure (workunit generator, database etc.) gets into trouble when we try to feed more than 40-50% of the project with these. We are working on a re-design of the FGRP workunits (also in perspective of the GPU application versions), but this will also take a while.

BM

BM

Neil Newell
Neil Newell
Joined: 20 Nov 12
Posts: 176
Credit: 169699457
RAC: 0

Thanks for the update; I

Thanks for the update; I appreciate that for this kind of work there's an awful lot of work that goes into making sure the science is as solid as possible (and that can't be rushed).

Now - where are my points????? (joking of course)

Maura
Maura
Joined: 4 Apr 13
Posts: 3
Credit: 4397840
RAC: 3632

RE: RE: So how come if we

Quote:
Quote:
So how come if we are using the same raw data are we going to find any other gravitational waves ?

Basically the 'sound' of a wave can vary - and no one has heard one yet - so different templates ( signal shapes ) and search methods are being tried. Such permutations have yet to be exhausted. Generally the noise is far greater than the signal of interest and so there is extra subtlety due to that. For that matter, no waves may ever be heard and that is one point of such investigations : to determine whether General Relativity is correct in that regard.

Cheers, Mike.

And if it is not? What if Quantum Gravity is no more real than tachyonic mass, QM and GR are irredeemably separate, string theory is wrong and Gravitational Waves don't exist?
Would we have fun, or just confusion?

PovAddict
PovAddict
Joined: 31 Mar 05
Posts: 45
Credit: 2508915
RAC: 0

For some reason I find it fun

For some reason I find it fun to run the last mile of a project/subproject/batch. More so if some people will temporarily drop this subproject because of the extra downloading: I get to (slightly) compensate for it.

Does "S6BucketLVE search progress" in the server status page show how many tasks are left for all this particular search to finish? (The "Workunits and tasks" section obviously doesn't, since that only shows tasks in the server's BOINC queue, and the data isn't necessarily all queued in BOINC already)

Bernd Machenschalk
Bernd Machenschalk
Moderator
Administrator
Joined: 15 Oct 04
Posts: 4265
Credit: 244921706
RAC: 16859

RE: Does "S6BucketLVE

Quote:
Does "S6BucketLVE search progress" in the server status page show how many tasks are left for all this particular search to finish?

More precisely it shows the number of workunits remaining to be generated. Each such workunit will give two tasks to be sent out for processing by the clients. The workunit will be finished when there was found a "canonical result" from the results returned. "Work still remaining" shows the prediction of when we will run out of tasks to send out; it usually takes much longer until the last workunit of the search is finished (~6 weeks). This prediction is based on averaged processing rate over the whole run, in this case including e.g. a couple of "challenges" that were ran. Currently the processing rate is below average, so I would expect the remaining work to last a few days longer.

BM

BM

Mike Hewson
Mike Hewson
Moderator
Joined: 1 Dec 05
Posts: 6534
Credit: 284700169
RAC: 113726

RE: And if it is not? What

Quote:
And if it is not? What if Quantum Gravity is no more real than tachyonic mass, QM and GR are irredeemably separate, string theory is wrong and Gravitational Waves don't exist?
Would we have fun, or just confusion?


Probably both! At least "trouble at t'mill" :-)

Quote:
[ASIDE] QM + GR : My guess is data from behaviours in high gravitational fields would show the way, as that's the domain where many concepts collide. Personally I think we need a new form of math, even new number 'types' to make headway. The base problem is that we have no unified view cleanly encompassing both sharp transitions with smooth evolutions ( eg. which is why we have the doubtful prospect of wave functions 'collapsing' to achieve a measurement ). Rene Thom had a go at this with 'catastrophe theory' but it never generalised sufficiently.


Of course maybe the universe is not explicable from any viewpoint within. All that we know, or could ever know, may be nought but a layer of a palimpsest ( an overwritten document ). But that'd kinda suck the fun out of inquiring projects like this, so I'll optimistically go for explicable. :-0

Cheers, Mike.

I have made this letter longer than usual because I lack the time to make it shorter ...

... and my other CPU is a Ryzen 5950X :-) Blaise Pascal

PovAddict
PovAddict
Joined: 31 Mar 05
Posts: 45
Credit: 2508915
RAC: 0

16 jun 2013 01:51:54 ART No

16 jun 2013 01:51:54 ART No work is available for Gravitational Wave S6 LineVeto search (extended)
16 jun 2013 01:51:54 ART No work is available for Gravitational Wave S6 Directed Search (CasA)

EDIT: bah nevermind, just got some.

Tugrul_512bit
Tugrul_512bit
Joined: 18 Jun 13
Posts: 4
Credit: 69811
RAC: 0

I'm happy to share my Gflops

I'm happy to share my Gflops for this kind of research. I have a problem though, my pitcairn hd7870 has 20-compute units and E@H uses only 15-16 of them(about maximum %70 gpu usage even with overclocked CPU and lowered CPU usage.) Other projects such as Milkyway@home uses %100 GPU.

Opencl Enthusiast

Tugrul_512bit
Tugrul_512bit
Joined: 18 Jun 13
Posts: 4
Credit: 69811
RAC: 0

Just made gpu utilization 0.2

Just made gpu utilization 0.2 rather than 1.0. This made 0.5CPU+0.2GPU for 5 simultaneous workloads and increased gpu-usage to %93. Can this damage hardware?

Opencl Enthusiast

Holmis
Joined: 4 Jan 05
Posts: 1118
Credit: 1055935564
RAC: 0

RE: Just made gpu

Quote:
Just made gpu utilization 0.2 rather than 1.0. This made 0.5CPU+0.2GPU for 5 simultaneous workloads and increased gpu-usage to %93. Can this damage hardware?

Probably only if you have insufficient cooling and the increased load heats up the card to much.
I would also recommend that you test more values for the GPU utilization factor, like 0.5 for x2, 0.33 for x3 and maybe also 0.25 for x4 to see if you get better load than running x5. There are other parts of the computer like the PCIe-bus, system RAM and so on that can be overloaded when running too many tasks at once and thus lowering the overall performance.

PovAddict
PovAddict
Joined: 31 Mar 05
Posts: 45
Credit: 2508915
RAC: 0

RE: I'm happy to share my

Quote:
I'm happy to share my Gflops for this kind of research. I have a problem though, my pitcairn hd7870 has 20-compute units and E@H uses only 15-16 of them(about maximum %70 gpu usage even with overclocked CPU and lowered CPU usage.)


Keep in mind that there is no GPU application for the Gravitational Wave search, which is what this thread is about.

Tugrul_512bit
Tugrul_512bit
Joined: 18 Jun 13
Posts: 4
Credit: 69811
RAC: 0

Thanks, Im looking for best

Thanks, Im looking for best ratios and lowering frequencies a bit.

Opencl Enthusiast

GALAXY-VOYAGER
GALAXY-VOYAGER
Joined: 24 Oct 12
Posts: 32
Credit: 92959
RAC: 0

What does this mean for The

What does this mean for The Future of the einstein@home Project itself ?

GALAXY-VOYAGER
GALAXY-VOYAGER
Joined: 24 Oct 12
Posts: 32
Credit: 92959
RAC: 0

Are you participoating in

Are you participoating in seti@home ? ..... SETI is the First Project I became involved with, and have Joined EINSTEIN and some other Projects since that time. Besides seti@home, einstein@home is my preferred Project. I find SETI Very Interesting. Others that I use and I find interesting is constellation@home, cosmology@home, and rosetta@home (although, Rosetta is not a Space Related Project).

Cheers,

Bernd Machenschalk
Bernd Machenschalk
Moderator
Administrator
Joined: 15 Oct 04
Posts: 4265
Credit: 244921706
RAC: 16859

RE: What does this mean for

Quote:
What does this mean for The Future of the einstein@home Project itself ?

I'm not sure what exactly you mean by 'this' and thus what precisely you are asking.

For a few weeks Einstein@Home will not be searching for Gravitational Waves.

The changes we recently made to the Radio Pulsar search and the Gamma-Ray Pulsar search are meant to ensure that the project as a whole will remain functional and healthy while we are developing, testing and tuning the next Gravitational Wave search over at Albert@Home. When we are sufficiently confident that we do the most sensitive search possible with your precious contributed computing power, we will start this new search here at Einstein@Home.

BM

BM

Logforme
Logforme
Joined: 13 Aug 10
Posts: 332
Credit: 1714373961
RAC: 0

How many people seem to read

How many people seem to read this thread subject:
"Gravitational Wave" "ENDING" .. OMG the project is DYING

Lesson from this: You can never be TOO clear in a news post :)

Mike Hewson
Mike Hewson
Moderator
Joined: 1 Dec 05
Posts: 6534
Credit: 284700169
RAC: 113726

RE: How many people seem to

Quote:

How many people seem to read this thread subject:
"Gravitational Wave" "ENDING" .. OMG the project is DYING

Lesson from this: You can never be TOO clear in a news post :)


True enough. :-)

ASIDE : One ( recurrent ) issue here at E@H is that many visiting here don't have English as their primary language, and even if so may not have sufficiently sophisticated vocabulary/semantics to grasp the key points presented. This reminds me of a sign in an army barracks, used to temporarily house immigrants to DownUnda after WWII - 'ALL BEDS MUST BE MADE UP AS LAID DOWN IN STANDING ORDERS'. This triggered an array of virtually every conceivable 3D arrangement of bedding that you could think of, including one chap who put his bed upon the roof ..... :-)

Cheers, Mike.

( edit ) Here's a laugh : take the order as above in English, run it through Google translate to French then Spanish then German then Italian and finally back to English and you get 'ALL ROOMS MUST BE LAYING DOWN' !!! :-) :-)

Maybe this could be a new Olympic sport, if it isn't already.

I have made this letter longer than usual because I lack the time to make it shorter ...

... and my other CPU is a Ryzen 5950X :-) Blaise Pascal

Filipe
Filipe
Joined: 10 Mar 05
Posts: 175
Credit: 365519120
RAC: 45514

22 Work units to go

22 Work units to go