NVIDIA drivers present but no GPUs found (Ubuntu 14.04, GTX970)

You haven't described what the beginning troubles were exactly and i could not find any earlier posts - i guess they are the same issue?

I have had several years running Ubuntu ok with nVidia, without too much difficulty, however i'm reasonably confident with running at the command line.

Others may have a different approach, but i
+ never use GPU drivers from registry
+ never upgrade OS in place
+ read the updates before applying them.
+ do not install optional updates unless a benefit
+ always apply security updates
+ keep to LTS versions of Ubuntu.
+ keep a rescue boot usb handy
+ don't assume a driver works well for all nvidia cards.
+ don't assume the latest driver is more reliable than the one you are currently using. The E@H forums usually have good advice, the release notes and the README usually give a few clues.

I haven't updated nVidia drivers for a while but currently i have been running 349.12 for over 6 months.

Your version: 346.35 had a release date of 2015.1.16 and was the first to support the gtx970 from what i can tell - i have not seen any discussion good or bad on this release, however i would reiterate other advice given of getting a newer driver.

You will need to uninstall and purge the old driver (apt-get ...) and that may have an impact of the system stability and so reinstall from scratch may be easier. If you install a nVidia supplied driver manually you must not use repository version.

Latest 64 bit release 352.41 has the location of the latest driver.

There is a very good GPU recognition post here which helped me GPUs : Debian/Ubuntu/Mint/Derivatives - GPU recognition fixes which is worth reading especially checking the delay settings.

Good luck.

So you download the driver package from nvidia and build your own?

I did that for fglrx drivers (AMD) when I first started playing with a HD 7770 card. With the distro I use (not ubuntu or derivatives), I've never had to do it with nvidia as the repo supplied drivers and libs have always worked. When I first wanted to try later versions of fglrx, I was able to use the repo supplied drivers as the packaging issues had been fixed.

I currently have over 20 hosts with various nvidia cards. The version I'm running is 331.49. They run fine and I tend not to update unless there seems to be a reported performance gain by doing so. I do test this (relatively infrequently) and about a month or two ago I decided to buy a GTX 750Ti and try it out with the latest versions of everything for my distro of choice. I don't remember exactly what driver version it was but it was quite recent.

My distro packages the driver separately from what it calls the 'CUDA and OpenCL high performance computing libs' and there is a note saying that these libs are not required for 'normal' use. I actually feel rather special being NOT 'normal' :-). The libs are there in the repo - you just have to make sure you install them. So I made sure they were installed for the test using the GTX 750Ti.

When I tried to start crunching with the new GPU, I got pretty much the same result as reported by the OP - the driver was recognised but no usable GPUs were found. I double checked everything to no avail. So I reinstalled the OS using the kernel and drivers from May 2014 (exactly what is on all other nvidia hosts) and the new machine started crunching straight away.

On further investigation, I found this thread on my distro's forums which I believe is why I couldn't get my card running with the latest driver - a missing module called nvidia-uvm that is now needed for GPU computing. If you look for posts by "TerryN", the person who packages the nvidia drivers you will find this statement

Of course, that explains my problem - I just have to wait until TerryN works out how he will deal with the nvidia-uvm module - but I know nothing about the situation with ubuntu. All I can suggest is that the OP does some digging in the ubuntu world to see if the same problem exists there. For what nvidia-uvm is and when it was added, check out the 5th bullet point on this page,

Exactly! With 20+ nvidia and 30+ AMD, I need a quick and painless method to get a host fully configured and working. New installs are one thing but there is a significant saving in having simple procedures for coping with hardware reconfiguration and hardware failure. I keep multiple versions (approx 6 months apart) of my distro's repos on a 1TB USB drive, together with multiple OS versions that can be booted as live media (much faster than a CD). It goes back for the last couple of years so I can very quickly go back to a known good version if the very latest has an issue. Once I found the problem with my new 750Ti and the very latest, it took about 20 mins to reinstall the OS and update to a known good point in time and get back to crunching - quick and painless :-).

Yesterday, my control script notified me of an unresponsive host. On trying to reboot that host, there was no hard disk being detected by the BIOS. Pulled the plug for about 5 mins and tried again. Disk was identified but SMART status was BAD. It was a 40GB IDE from about 2004. Hooked up another old IDE drive to the second plug on the IDE cable and booted the live USB I use for installs and recovery.

Ran fsck on the partitions on the BAD drive. They checked out OK so I created suitable partitions on the 2nd drive, mounted /home partitions of each drive on /media/sdax and /media/sdbx and simply copied the entire contents of /media/sdax to /media/sdbx. There were no copying errors reported so I reinstalled the OS on the root partition of the replacement drive, installed drivers (HD 7850) and applied all extras and updates using my 'date-of-choice' repo copy. After shutting down and removing the BAD drive, I was able to reboot the machine on the replacement drive and immediately restart crunching (4xGPU, 2xCPU) from all the saved checkpoints. Not a single task was lost and the entire exercise took less than two hours.

This is really needed when crunching is an adjunct to the main use of the machine. When the machine only does crunching, I prefer to get versions straight from Berkeley and install them directly in a subdirectory of the home directory - /home/gary/BOINC, with everything owned by 'gary'. I don't allow the boinc daemon to autostart. There's an icon on the desktop to click to start the daemon and another one to stop it. The only time machines are restarted is when there is a problem and more often than not you may want to check things after a restart without having the boinc daemon doing its own thing. A big plus for me is the virtually instant version change capability. Stop BOINC, overwrite boinc, boinccmd and boincmgr with new versions, restart BOINC. The new versions are on the 1TB USB drive which automounts when plugged it.

I like 'farm' :-).

Yes indeed!! You need to read carefully as important info can be quite brief. For example, in the bullet point I mentioned, there is this little bit

... This kernel module provides support for the new Unified Memory feature in an upcoming CUDA release. In other words, the uvm module is already here but wont be needed until some later driver version incorporates an update to CUDA which will then be able to use this module. This is presumably why I can use this driver version with a 750Ti but without actually having a uvm module. I want to try a recent driver with the CUDA changes that require the uvm module because this may well improve performance for Maxwell1 GPUs as well as Maxwell2. If nvidia-uvm doesn't appear soon in my distro's repo, I'll probably build a driver from the stuff on the nvidia website (out of curiosity) :-).