Arecibo graphic question

Neil
Neil
Joined: 29 May 12
Posts: 8
Credit: 72182
RAC: 0
Topic 196481

I was wondering about the power spectrum in the upper right hand corner of the Arecibo graphic.. I had one task recently that had a power spike on the left hand side of the power scale.. Can anyone explain what that might represent?.. ty..

Bikeman (Heinz-Bernd Eggenstein)
Bikeman (Heinz-...
Moderator
Joined: 28 Aug 06
Posts: 3522
Credit: 692135122
RAC: 36029

Arecibo graphic question

Quote:
I was wondering about the power spectrum in the upper right hand corner of the Arecibo graphic.. I had one task recently that had a power spike on the left hand side of the power scale.. Can anyone explain what that might represent?.. ty..

Hi!

The bars on the left hand side represent lower frequencies, that's about all that can be said about this observation. It could be interference from man-made radio sources, or it could be a real astrophysical source, e.g. some of the pulsars found in the search so far have spin frequencies down to 0.7 Hz.

(see our overview, for exmple, of recent Arecibo findings, http://einstein.phys.uwm.edu/radiopulsar/html/BRP4_discoveries/
The observed frequency in Hz would be close to the value Pbary/1000.

Cheers
HB

Neil
Neil
Joined: 29 May 12
Posts: 8
Credit: 72182
RAC: 0

ty, for your help.. i am just

ty, for your help.. i am just a layman who has always been interested in the heavens, and the mysteries they propose.. again, ty for your explanation.. i appreciate it very much..

Denis Puhar, dr. med.
Denis Puhar, dr...
Joined: 5 Nov 09
Posts: 36
Credit: 7006583
RAC: 0

Hi! I'm also a layman, but

Hi!

I'm also a layman, but trying to learn as much as possible. Said otherwise:

I think that at least to a degree I understand the principle (NON-MATHEMATICAL!) behind what is being computed and searched for in the WUs, no matter if we are talking about (yet) theoretical gravitational waves or searching for pulsars in particular point in space.

And in the latter subject I'm especially interested in. Why?

Because I primarily crunch the GPU cuda BRP4 (Arecibo) WUs.

I'm interested into the meaning of the POWER SPECTRUM in those WUs (significance of spikes, lack of them, different heights of the bars or even the number of spikes you see there during the progress of the WU itself - most of the time: NONE, sometimes one or two, VERY RARELY a dozen or more, etc...,).

I excuse myself, if this has been already explained, but till now I didn't find anything at all.

This is one issue. The 2. is the stderr output you can observe, after the WU has been completed.

On what figures should a laymen focus? I ask this, because for a non-mathematician ALL of the data with figures seem virtually the same in EVERY WU processed.

I'm not looking for an explanation (because I wouldn't understand it anyway), but it would be really helpful, to know at what figures in the stderr output you should pay attention to.

EXAMPLE: I know, that in most of the WUs mentioned above there won't be a possible pulsar (in that SMALL particular point in space you were searching for), but if there is EVEN a REMOTE CHANCE, that you've found something, could please someone tell me which figures in the result will ALERT a cruncher to this possibility (even if it is a remote one).

Thank You in advance.

Denis

P.S.
I forgot earlier: Could even the sole presence of many white bars in the POWER spectrum possibly (!) indicate more significance as usual?

“A little knowledge is a dangerous thing. So is a lot.” - Albert EINSTEIN

Holmis
Joined: 4 Jan 05
Posts: 1118
Credit: 1055935564
RAC: 0

I'll take a stab at an

I'll take a stab at an explanation of how I think this works. If someone more knowledgeable finds error please do enlighten me! =)

I seem to remember an explanation somewhere on these message boards of the "Power spectrum" to be a real time representation of the signals/noise in the data as it's being processed. That is, higher bars would indicate some type of signal, but before this can be said to be a signal from a pulsar the data has to be post-processed back at the project together with data from multiple WUs.

Edit: Further down the page here in the Cafe I found this post from Mike about the Power spectrum.

Now over to the stderr output.
I believe stderr stands for "Standard error" and the output is a log of messages from the application that can be used to diagnose errors.

I've taken a part of a log from one of the WUs that you have processed and if we break it down I would say that this is what we get:

Quote:

7.0.33

Activated exception handling...
[19:52:42][8228][INFO ] Starting data processing...
[19:52:42][8228][INFO ] CUDA global memory status (initial GPU state, including context):
------> Used in total: 131 MB (893 MB free / 1024 MB total) -> Used by this application (assuming a single GPU task): 0 MB
[19:52:42][8228][INFO ] Using CUDA device #0 "GeForce GTX 550 Ti" (192 CUDA cores / 744.96 GFLOPS)
[19:52:42][8228][INFO ] Version of installed CUDA driver: 5000
[19:52:42][8228][INFO ] Version of CUDA driver API used: 3020


The part of the log above contains information about the Boinc version and the resource used to process this WU, in this case a "GeForce GTX 550 Ti".

Then follows some information about the data which is to be processed and at the end of this block is the search parameters used.

Quote:
[19:52:42][8228][INFO ] Checkpoint file unavailable: status.cpt (No such file or directory).
------> Starting from scratch...
[19:52:42][8228][INFO ] Header contents:
------> Original WAPP file: ./p2030.20110116.G193.64-04.05.C.b4s0g0.00000_DM87.20
------> Sample time in microseconds: 65.4762
------> Observation time in seconds: 274.62705
------> Time stamp (MJD): 55578.142897896694
------> Number of samples/record: 0
------> Center freq in MHz: 1214.289551
------> Channel band in MHz: 0.33605957
------> Number of channels/record: 960
------> Nifs: 1
------> RA (J2000): 60026.3430004
------> DEC (J2000): 150230.943399
------> Galactic l: 0
------> Galactic b: 0
------> Name: G193.64-04.05.C
------> Lagformat: 0
------> Sum: 1
------> Level: 3
------> AZ at start: 0
------> ZA at start: 0
------> AST at start: 0
------> LST at start: 0
------> Project ID: --
------> Observers: --
------> File size (bytes): 0
------> Data size (bytes): 0
------> Number of samples: 4194304
------> Trial dispersion measure: 87.2 cm^-3 pc
------> Scale factor: 0.0013764
[19:52:43][8228][INFO ] Seed for random number generator is 1168043486.
[19:52:44][8228][INFO ] Derived global search parameters:
------> f_A probability = 0.08
------> single bin prob(P_noise > P_thr) = 1.32531e-008
------> thr1 = 18.139
------> thr2 = 21.241
------> thr4 = 26.2686
------> thr8 = 34.6478
------> thr16 = 48.9581

Quote:
[19:52:44][8228][INFO ] CUDA global memory status (GPU setup complete):
------> Used in total: 335 MB (689 MB free / 1024 MB total) -> Used by this application (assuming a single GPU task): 204 MB


Above is some more memory information after the GPU setup is complete.

Then follows information about the actual processing of the WU and we can see that it checkpoints one time before completing.

Quote:
[19:54:42][8228][INFO ] Checkpoint committed!
[19:56:05][8228][INFO ] Statistics: count dirty SumSpec pages 441 (not checkpointed), Page Size 1024, fundamental_idx_hi-window_2: 329052
[19:56:05][8228][INFO ] Data processing finished successfully!

Then the whole thing repeats seven times as one WU consists of 8 individual tasks.
If something where to go wrong during processing then hopefully this log will contain information about this to help with diagnosing the error.

The actual results of the data processing is stored in a separate file and uploaded to the project after the WU is completed and the contents of that file is not shown in any log.

Mike Hewson
Mike Hewson
Moderator
Joined: 1 Dec 05
Posts: 6550
Credit: 288782830
RAC: 82439

RE: Edit: Further down the

Quote:
Edit: Further down the page here in the Cafe I found this post from Mike about the Power spectrum.


Yup, it's essentially a running snapshot of the power at each frequency, and thus easily polluted at low frequencies. In any cased de-dispersion ( accounting for waveform distortion or 'chirping' because of free electrons in the intervening space to source ) has to occur as well. You see this as 'Trial dispersion measure: 87.2 cm^-3 pc' which is a guess at that effect, hence our parameter space to search has dispersion measure as an axis. So if you take this result the 'DM channel' is mentioned.

HTH :-)

Cheers, Mike.

I have made this letter longer than usual because I lack the time to make it shorter ...

... and my other CPU is a Ryzen 5950X :-) Blaise Pascal

Denis Puhar, dr. med.
Denis Puhar, dr...
Joined: 5 Nov 09
Posts: 36
Credit: 7006583
RAC: 0

Hi! I'm very thankful for

Hi!

I'm very thankful for your methodical and simple to understand explanation.

As I thought myself (but was far from being sure about it) the 'role' of the POWER SPECTRUM is the 'real time' graphic representation of signal/noise ratio.

So if you are correct (and my thoughts on this are the same) I can draw only one conclusion out of this:

If an WU intermittently shows high activity in the Power spectrum then it is somehow more 'promising' in finding something as opposed if there was no activity at all.

That is a conclusion based merely on pure logic and data I have avaliable (almost none). So ANYBODY with more knowledge, PLEASE correct me, if this assumption is wrong.

In the case of the stderr output I'm really grateful to you. Most of the things you explained excellent and methodical I already knew (not all the details), but the part I DID NOT KNOW, helped me VERY much and saved me a lot of time (as I was searching for something, that did not exist at all :) namely:

Quote:
The actual results of the data processing is stored in a separate file and uploaded to the project after the WU is completed and the contents of that file is not shown in any log.

So again:

Thank you very much for your kind, quick response and explanation.

Denis

P.S.
Any other comments on the subject, written with very simplified words below, are welcomed.

Is there any way for the cruncher to know if a particular WU has higher chances of finding something significant than other?

P.P.S.
As I was writing, I did not notice, that beside Holmis I got also a reply from Mike Hewson. So thank you very much to BOTH of you.

“A little knowledge is a dangerous thing. So is a lot.” - Albert EINSTEIN

Mike Hewson
Mike Hewson
Moderator
Joined: 1 Dec 05
Posts: 6550
Credit: 288782830
RAC: 82439

RE: Is there any way for

Quote:
Is there any way for the cruncher to know if a particular WU has higher chances of finding something significant than other?


Alas no. Essentially there are too many confounding effects, and this is basically why computers are doing it. Pretty much all of the 'low hanging fruit' in this area of research has been plucked previously - the first pulsar ever discovered was analysed using paper tape. So now the search sophistication is matching the difficulty of the finds. But we are still plucking!! :-)

Cheers, Mike.

( edit ) Sorry, I didn't quite answer your primary question : you could get a detection in an area of the spectrum not noticeable by high bars. This is because

(a) the pulsar might be 'quiet' and yet we still manage to hear it, and

(b) after glancing at the screensaver code, the spectrum as displayed is normalised anyway ie. the heights of the bars are relative to the tallest.

( edit ) 'relative to the tallest.' : so specifically a high noise at some frequencies could scale down the graphics representation at other frequencies, even though the signal-to-noise ratio could well be quite favourable for signal detection at those other frequencies. So essentially the displayed power spectrum is really only something pretty to look at .... :-)

I have made this letter longer than usual because I lack the time to make it shorter ...

... and my other CPU is a Ryzen 5950X :-) Blaise Pascal

Denis Puhar, dr. med.
Denis Puhar, dr...
Joined: 5 Nov 09
Posts: 36
Credit: 7006583
RAC: 0

Thank you very much for your

Thank you very much for your response!

But it is not only that, but something MUCH MORE important!

As a physician I'm very much aware how hard it is sometimes to explain things to students or even lay-people, who are for example interested about a particular disease, a physiological process or something similar.

It happens quite often, that some things, you are asked about and which seem in your head as being 'common sense' and as a result of such mentality, your answer (as precise it might be to your fellow colleagues) contains too much complicated information, with the end-result, that the person, who was asking, is in the end:

1. 'Overflown' with too much conflicting data, which he does not understand;
2. More confused as he was before the question was even asked.

And I'm not even talking about 'rocket' science here, nor am I an 'academic' in my area. But nevertheless I MUST never forget, that I studied 7 years, to gain BASIC knowledge (because if my 'vocabulary' would not contain the sentence: 'I don't know.' sometimes, I'd be lying to myself and doing more harm than good).

I can only imagine, how someone with your knowledge, possesses much patience and pedagogic abilities (because in this scientific area is even a larger gap between lay-man and professionals, namely - said simplified - mathematics) to explain VERY COMPLEX things, to a wide 'public', with varying and diverse levels of education.

That is the MAIN reason, that I'm grateful for your time. It is hard enough to be a good physicist (or mathematician), but nothing in comparison with the skill of being able to explain difficult topics in a way, that the person, who is asking, knows at the 'end of the day' more (and becomes maybe even more curious), opposed to receiving lots of data, which (at least for him) is useless.

For me, that is the MOST SIGNIFICANT difference between an average and excellent teacher.

Regards.

Denis

P.S.
I excuse myself for being so long, as English is not my primary language and I'm sometimes almost struggling to find the right words.

“A little knowledge is a dangerous thing. So is a lot.” - Albert EINSTEIN

Fred J. Verster
Fred J. Verster
Joined: 27 Apr 08
Posts: 118
Credit: 22451438
RAC: 0

---[SNIPPED]--- RE: As

---[SNIPPED]---

Quote:


As a physician I'm very much aware how hard it is sometimes to explain things to students or even lay-people, who are for example interested about a particular disease, a physiological process or something similar.

It happens quite often, that some things, you are asked about and which seem in your head as being 'common sense' and as a result of such mentality, your answer (as precise it might be to your fellow colleagues) contains too much complicated information, with the end-result, that the person, who was asking, is in the end:

1. 'Overflown' with too much conflicting data, which he does not understand;
2. More confused as he was before the question was even asked.

And I'm not even talking about 'rocket' science here, nor am I an 'academic' in my area. But nevertheless I MUST never forget, that I studied 7 years, to gain BASIC knowledge (because if my 'vocabulary' would not contain the sentence: 'I don't know.' sometimes, I'd be lying to myself and doing more harm than good).

I can only imagine, how someone with your knowledge, possesses much patience and pedagogic abilities (because in this scientific area is even a larger gap between lay-man and professionals, namely - said simplified - mathematics) to explain VERY COMPLEX things, to a wide 'public', with varying and diverse levels of education.
---[Snipped]---

Well i haven't made it out of college with a degree, but I'm thinking
along the same lines.
Enlish not being my native language makes it even harder, but one seems
never too 'old' to learn new things if health plays ball.

Einstein@home uses a lot data from the Arecibo Observatory which I believe
has 1.42GHz as center frequency.
Is it a ramdom generator which picks the frequency being examend/computed?

Graphics are often 'eye candy' but graphic display of data can be accurate.

The discovery of Pulsars makes one wondering what kind of 'celestial
bodies' might be emitting some form of E.M.R. apart from visual or 'stars'?
There's a lot of room between a 10cm wave and a 10nm wave or sinus.

But I shouldn't derail the thread and stay on topic ;-)

Mike Hewson
Mike Hewson
Moderator
Joined: 1 Dec 05
Posts: 6550
Credit: 288782830
RAC: 82439

RE: Einstein@home uses a

Quote:
Einstein@home uses a lot data from the Arecibo Observatory which I believe
has 1.42GHz as center frequency.
Is it a ramdom generator which picks the frequency being examend/computed?


It might be simpler to think of 1.42GHz as 'photons of a certain energy' ( or within some defined band ) with the variation in time of the flux/number of those being received as our primary signal data. There is the famous connection between photon frequency ( as disclosed by experiments that demonstrate wave effects like diffraction and interference ) and photon energy ( as exhibited by the photo-electric effect or atomic electron transitions ) which Albert Einstein championed ( although presaged by Max Planck's treatment of radiation in cavities ) :

Energy = h * frequency

h = Planck's constant

More photons -> stronger signal, less photons -> weaker signal. The frequency mentioned in our work units is how rhythmic is the variation in that photon flux, not the choice of energy of the photons we choose to count. So presumably the ~1.42GHz is a technical choice related to how one designs, builds and operates an Aricebo dish.

You see, back at the pulsar there is this huge ( double ended ) photon 'hose' which pours out a massive amount of radiation at a wide range of energies. These are very unhealthy neighbourhoods to be around. Think microwave oven. Because of the spinning of the ( neutron ) star the hose is whipped around in a circle, and if the geometry is right with regards to it's pointing in 3D space combined with where we are then we get splashed intermittently by the photons from said hose. There are no doubt many of these objects that we will never be aware of as there are configurations aplenty where we never get wet. Some pulsars are labelled as 'intermittent' which is I think generally interpreted as the hose changing the plane of it's rotation ( due to the influence of nearby objects ) now and then. There are even 'binary pulsars', fascinating beasts, that have a pair of these guys ( each with their own hoses ) circling around each other and maybe sometimes blocking from us the other partner's signal .....

Think of your favorite lawn sprinkler ( or the usual lighthouse analogy ). So it's the frequency of the rotation of the sprinkler that we are seeking ie. how fast is this neutron star spinning? [ I'm skipping over a heap of arcane stuff I barely understand ... ]

I'm always a bit boggled by these objects : mass about that of the Sun ( give or take ), around 20km in diameter, a stupendous magnetic field, and some have been found to spin faster than a Ferrari's crankshaft at full throttle! :-)

Quote:
Graphics are often 'eye candy' but graphic display of data can be accurate.


Well if you see something sticking out a mile, away from the left/low frequency end, then don't let me crush your optimism! :-)

Cheers, Mike.

I have made this letter longer than usual because I lack the time to make it shorter ...

... and my other CPU is a Ryzen 5950X :-) Blaise Pascal

Comment viewing options

Select your preferred way to display the comments and click "Save settings" to activate your changes.