S5GCE, was: Beyond S5R6

log in

Advanced search

Message boards : Cruncher's Corner : S5GCE, was: Beyond S5R6

1 · 2 · 3 · 4 . . . 7 · Next
Author Message
DanNeely
Send message
Joined: 4 Sep 05
Posts: 1120
Credit: 188,372,979
RAC: 241,696
Message 102302 - Posted: 31 Jan 2010, 1:55:46 UTC - in response to Message 102301.

I'll focus on preparing the next GW search, so there won't be much progress on CUDA in February.



Can you tell us anything about it yet: S5R7 or S6R1? Similar to the current run, or significant differences?
____________
Profile Bernd Machenschalk
Volunteer moderator
Project administrator
Project developer
Avatar
Send message
Joined: 15 Oct 04
Posts: 3611
Credit: 128,438,221
RAC: 56,951
Message 102306 - Posted: 31 Jan 2010, 13:07:52 UTC - in response to Message 102302.
Last modified: 31 Jan 2010, 13:11:41 UTC

Can you tell us anything about it yet: S5R7 or S6R1? Similar to the current run, or significant differences?


The data taken in S6 so far is actually noisier than what we have of S5, so we'll stick to S5 data for now.

We're working on a new program that will combine a new, faster implementation of the FStat calculation with a new method called 'Global Correlation Transform' that will supersede the 'Hough Transform' used in the current HierarchicalSearch, resulting in the most sensitive search for continuous signals that we know, and still requiring much less computing power for the same parameter space.

We work on turning the program into a reliable BOINC application and on setting up a search design (i.e. how to split up the parameter space into workunits), which will result in a new workunit generator and possibly new data files.

With luck, we'll have that ready by the time we run out of S5R6 work. If we aren't, we'll insert a short run with the same application and data as we used on S5R5 and S5R6, but looking into higher frequencies. S5R5 went up to 1kHz, S5R6 covered 1kHz to 1.2kHz, the current data files allow to search up to 1.5kHz.

BM
Ver Greeneyes
Send message
Joined: 26 Mar 09
Posts: 140
Credit: 9,562,235
RAC: 0
Message 102307 - Posted: 31 Jan 2010, 14:37:11 UTC - in response to Message 102306.

We're working on a new program that will combine a new, faster implementation of the FStat calculation with a new method called 'Global Correlation Transform' that will supersede the 'Hough Transform' used in the current HierarchicalSearch, resulting in the most sensitive search for continuous signals that we know, and still requiring much less computing power for the same parameter space.

Wow, that sounds fascinating! It always amazes me how much brute force processing one can avoid by improving the algorithms employed.
Profile Bernd Machenschalk
Volunteer moderator
Project administrator
Project developer
Avatar
Send message
Joined: 15 Oct 04
Posts: 3611
Credit: 128,438,221
RAC: 56,951
Message 102312 - Posted: 31 Jan 2010, 18:09:06 UTC

discussion of what's to come after S5R6 has finished that started in ABP2 CUDA applications.

BM

Profile Bernd Machenschalk
Volunteer moderator
Project administrator
Project developer
Avatar
Send message
Joined: 15 Oct 04
Posts: 3611
Credit: 128,438,221
RAC: 56,951
Message 102451 - Posted: 10 Feb 2010, 9:41:38 UTC
Last modified: 10 Feb 2010, 11:26:12 UTC

We found that setting up a search (data, workunits) for the new code is pretty difficult given the little experience we do have with it. So most liekely we'll have a short (~2 month) "engineering run" with a code that combines the new GCT with the "traditional" FStat code, analyzing the whole S5 data (first half was used in S5R1-3, second half in S5R4-6), and looking at a parameter space that has already been covered (partly) by S5R3 and S5R5, so we can compare the results and get a better feeling for the workunit setup.

BM

DanNeely
Send message
Joined: 4 Sep 05
Posts: 1120
Credit: 188,372,979
RAC: 241,696
Message 102880 - Posted: 7 Mar 2010, 21:04:24 UTC

With only ~10 days of S5R6 data remaining can you give us an update on where the next run stands?
____________

Profile Bernd Machenschalk
Volunteer moderator
Project administrator
Project developer
Avatar
Send message
Joined: 15 Oct 04
Posts: 3611
Credit: 128,438,221
RAC: 56,951
Message 102881 - Posted: 7 Mar 2010, 21:09:54 UTC

Not much news. The run will probably be named S5GCE (S5 data Global Correlation Engineering run) and preparation and testing is underway internally. Looks like we're well in the plan. At current processing rate S5R6 will run out of (new) work on March 18th, we should be able to run the first larger tests on the main project around the next weekend.

BM

Profile Bernd Machenschalk
Volunteer moderator
Project administrator
Project developer
Avatar
Send message
Joined: 15 Oct 04
Posts: 3611
Credit: 128,438,221
RAC: 56,951
Message 102942 - Posted: 12 Mar 2010, 16:52:21 UTC
Last modified: 12 Mar 2010, 17:01:39 UTC

Testing is underway.

Applications are currently x86 only. I think we will continue to suppoert Mac OS X PPC, but the Apps for this platform might be delayed by a few days.

For this run we'll use the database for 'locality scheduling' communication between workunit generator and scheduler, instead of the communication files being used up to S5R6. AFAIK E@H is the only project using locality scheduling, so though being a medium change this doesn't affect other projects. In the mid term this should improve the overall speed of locality scheduling and the handling of data files both on the server and client side. This changes is currently giving us most of the headache.

BM

Profile tullio
Send message
Joined: 22 Jan 05
Posts: 1953
Credit: 2,910,075
RAC: 31,769
Message 102944 - Posted: 12 Mar 2010, 17:38:37 UTC

I've read in an Italian site of BOINC.Italy that the Einstein server is going to be shutdown this weekend to allow an OS upgrade. Is this true?
Tullio
____________

Profile Bernd Machenschalk
Volunteer moderator
Project administrator
Project developer
Avatar
Send message
Joined: 15 Oct 04
Posts: 3611
Credit: 128,438,221
RAC: 56,951
Message 102945 - Posted: 12 Mar 2010, 20:53:00 UTC - in response to Message 102944.

I've read in an Italian site of BOINC.Italy that the Einstein server is going to be shutdown this weekend to allow an OS upgrade. Is this true?
Tullio


That's news to me. Not that I know of. Maybe some other project server, or maybe someone completely misunderstood my post of March 7 here in this thread.

BM
Profile cenit
Send message
Joined: 25 Nov 05
Posts: 3
Credit: 5,401,638
RAC: 3,865
Message 102947 - Posted: 12 Mar 2010, 23:11:45 UTC - in response to Message 102945.
Last modified: 12 Mar 2010, 23:12:18 UTC

I've read in an Italian site of BOINC.Italy that the Einstein server is going to be shutdown this weekend to allow an OS upgrade. Is this true?
Tullio


That's news to me. Not that I know of. Maybe some other project server, or maybe someone completely misunderstood my post of March 7 here in this thread.

BM

about the wrong news on the Boinc.Italy website: I'm sorry, tullio, admins, maybe someone in the news/translation team made something wrong. I have removed the part about the OS upgrade, now our news is more faithful to the original english one that there's in the homepage (Project News March 9, 2010)...
____________
_badger
Avatar
Send message
Joined: 8 Mar 05
Posts: 12
Credit: 4,623,547
RAC: 0
Message 102948 - Posted: 13 Mar 2010, 8:01:52 UTC - in response to Message 102306.

Can you tell us anything about it yet: S5R7 or S6R1? Similar to the current run, or significant differences?


The data taken in S6 so far is actually noisier than what we have of S5, so we'll stick to S5 data for now.

We're working on a new program that will combine a new, faster implementation of the FStat calculation with a new method called 'Global Correlation Transform' that will supersede the 'Hough Transform' used in the current HierarchicalSearch, resulting in the most sensitive search for continuous signals that we know, and still requiring much less computing power for the same parameter space.

We work on turning the program into a reliable BOINC application and on setting up a search design (i.e. how to split up the parameter space into workunits), which will result in a new workunit generator and possibly new data files.

With luck, we'll have that ready by the time we run out of S5R6 work. If we aren't, we'll insert a short run with the same application and data as we used on S5R5 and S5R6, but looking into higher frequencies. S5R5 went up to 1kHz, S5R6 covered 1kHz to 1.2kHz, the current data files allow to search up to 1.5kHz.

BM


For those curious in the new 'Global Correlation Transform' run, this(.pdf) bit of light reading should clear things up. </he laughs>

Which poor developer found this on their desk??
DanNeely
Send message
Joined: 4 Sep 05
Posts: 1120
Credit: 188,372,979
RAC: 241,696
Message 102950 - Posted: 13 Mar 2010, 19:33:27 UTC - in response to Message 102948.


For those curious in the new 'Global Correlation Transform' run, this(.pdf) bit of light reading should clear things up. </he laughs>

Which poor developer found this on their desk??


Speaking as a developer (not on E@H) there are far worse things than to be given an overly technical spec written by people who know exactly what they want. As usual Dilbert captures the normal requirement acquisition problem perfectly.


For anyone who doesn't want to read through the entire paper, or try and figure out what it means: figure 2 sums up what it means in terms of the likelihood of making a detection. The signal strength that's almost certain to be detected with the new approach is an order of magnitude fainter than with the old one: 1 part in 10^-23 vs 1 part in 10^-22. The point at which the detection probability approaches zero remains roughly fixed at 1 part in 2 * 10^-24. The improvement comes primarily by reducing the width of the zone where a signal might be detected or might not be.


____________
Profile Bernd Machenschalk
Volunteer moderator
Project administrator
Project developer
Avatar
Send message
Joined: 15 Oct 04
Posts: 3611
Credit: 128,438,221
RAC: 56,951
Message 102961 - Posted: 16 Mar 2010, 8:26:17 UTC

We stumbled over a major problem with our new Windows S5GCE App. Looks like that we'll run out of S5R6 work before we can start S5GCE, i.e. for some days we will send out only ABP work.

BM

Profile Bernd Machenschalk
Volunteer moderator
Project administrator
Project developer
Avatar
Send message
Joined: 15 Oct 04
Posts: 3611
Credit: 128,438,221
RAC: 56,951
Message 102973 - Posted: 18 Mar 2010, 16:02:19 UTC

Originally we planned to generate just a few S5GCE tasks and then wait until they were reported back.

However the project behaved quite sluggish after we ran out of S5R6 work, so we decided to issue S5GCE work right away to keep the client busy and stop them from hammering on the server.

The Application had been tested as far as we could on the test project, but not yet 'in the wild' where it could behave quite differently.

BM

Alinator
Send message
Joined: 8 May 05
Posts: 927
Credit: 9,352,143
RAC: 0
Message 102974 - Posted: 18 Mar 2010, 16:38:58 UTC - in response to Message 102973.

Originally we planned to generate just a few S5GCE tasks and then wait until they were reported back.

However the project behaved quite sluggish after we ran out of S5R6 work, so we decided to issue S5GCE work right away to keep the client busy and stop them from hammering on the server.

The Application had been tested as far as we could on the test project, but not yet 'in the wild' where it could behave quite differently.

BM


Hmmm...

Interesting. In any event I've picked up a few, including one on one of my AMD old timers (251595}.

So far so good there. After one and a half hours of run time I'm projecting about a 300 Ksec execution time (which is decreasing at this point).

Alinator
Ver Greeneyes
Send message
Joined: 26 Mar 09
Posts: 140
Credit: 9,562,235
RAC: 0
Message 102976 - Posted: 18 Mar 2010, 18:18:27 UTC - in response to Message 102974.

I'm having a lot of trouble actually getting these tasks at the moment. Very rarely, one of the required files will begin downloading - at which point it finishes quickly - but most of the time they just sit there until BOINC decides to try the next file in the list. Is this simply a result of server-wide performance issues, or an actual problem with new code?

Michael Karlinsky
Avatar
Send message
Joined: 22 Jan 05
Posts: 896
Credit: 21,282,416
RAC: 18,951
Message 102977 - Posted: 18 Mar 2010, 19:45:29 UTC - in response to Message 102976.

I'm having a lot of trouble actually getting these tasks at the moment. Very rarely, one of the required files will begin downloading - at which point it finishes quickly - but most of the time they just sit there until BOINC decides to try the next file in the list. Is this simply a result of server-wide performance issues, or an actual problem with new code?


Same here.

According to server status, only one out of five mirrors are up.


Michael
____________
Team Linux Users Everywhere
Profile Bernd Machenschalk
Volunteer moderator
Project administrator
Project developer
Avatar
Send message
Joined: 15 Oct 04
Posts: 3611
Credit: 128,438,221
RAC: 56,951
Message 102978 - Posted: 18 Mar 2010, 19:45:49 UTC - in response to Message 102976.
Last modified: 18 Mar 2010, 19:46:18 UTC

I'm having a lot of trouble actually getting these tasks at the moment. Very rarely, one of the required files will begin downloading - at which point it finishes quickly - but most of the time they just sit there until BOINC decides to try the next file in the list. Is this simply a result of server-wide performance issues, or an actual problem with new code?


This seems to be a common problem on our download mirrors, neither with the apps nor the main server.

BM
Profile Bernd Machenschalk
Volunteer moderator
Project administrator
Project developer
Avatar
Send message
Joined: 15 Oct 04
Posts: 3611
Credit: 128,438,221
RAC: 56,951
Message 102979 - Posted: 18 Mar 2010, 19:54:08 UTC - in response to Message 102977.

According to server status, only one out of five mirrors are up.


It's not that the mirrors are down, they just appear to be overloaded.

BM
1 · 2 · 3 · 4 . . . 7 · Next

Message boards : Cruncher's Corner : S5GCE, was: Beyond S5R6


Home · Your account · Message boards

This material is based upon work supported by the National Science Foundation (NSF) under Grants PHY-1104902, PHY-1104617 and PHY-1105572 and by the Max Planck Gesellschaft (MPG). Any opinions, findings, and conclusions or recommendations expressed in this material are those of the investigators and do not necessarily reflect the views of the NSF or the MPG.

Copyright © 2016 Bruce Allen