CUDA runtime or driver is not supported

Support for issues specific to the Linux operating systems.

CUDA runtime or driver is not supported

Postby xhsh » Thu Feb 24, 2011 12:49 am

Hello, all,

I am testing the examples coming with cula. The compilation is successful. However, when I run "gesv", I get the following messages:

-------------------
SGESV
-------------------
Allocating Matrices
Initializing CULA
CUDA runtime or driver is not supported


However, I do not have any problems when I call the cublas functions coming with the cuda-toolkit. Could anybody tell me what is wrong, please?
xhsh
 
Posts: 8
Joined: Wed Feb 23, 2011 5:42 pm

Re: CUDA runtime or driver is not supported

Postby john » Mon Feb 28, 2011 2:40 pm

Please be sure that your CUDA version and CUDA driver are both updated to 3.2 and that your CULA installation is R10.
john
Administrator
 
Posts: 587
Joined: Thu Jul 23, 2009 2:31 pm

Re: CUDA runtime or driver is not supported

Postby hcula8 » Tue Aug 23, 2011 9:31 pm

Hello,

I get the same error while trying to run a program for calling sgesv. My Cuda driver version is 260.19.36 and the CUDA version is 3020, and I have CULA R11. Shouldnt it work fine with R11 ? or should I revert back to R10 ?
hcula8
 
Posts: 5
Joined: Fri Aug 19, 2011 9:15 am

Re: CUDA runtime or driver is not supported

Postby john » Wed Aug 24, 2011 10:32 am

That should be correct, though you'll want to check that you're not somehow mistakenly loading up a different CUDA runtime from somewhere else on your system.
john
Administrator
 
Posts: 587
Joined: Thu Jul 23, 2009 2:31 pm

Re: CUDA runtime or driver is not supported

Postby hcula8 » Wed Aug 24, 2011 11:32 am

Running the version check example gives the following output.

- CULA: 11.0
- CUDA Runtime: 3.2
- CUDA Driver: 3.2
- CUBLAS Runtime: 3.2

So my program should also be loading the 3.2 version of CUDA runtime. Could there be any other reason for the error ?
hcula8
 
Posts: 5
Joined: Fri Aug 19, 2011 9:15 am

Re: CUDA runtime or driver is not supported

Postby john » Wed Aug 24, 2011 1:18 pm

I don't see anything that would overtly cause version check to succeed and sgesv to fail. I'd like to ask for your LD_LIBRARY_PATH and if possible to try a CUDA 4 driver (you can keep toolkit 3.1 and R11 if that is your preference).
john
Administrator
 
Posts: 587
Joined: Thu Jul 23, 2009 2:31 pm

Re: CUDA runtime or driver is not supported

Postby hcula8 » Wed Aug 24, 2011 11:01 pm

my LD_LIBRARY_PATH=/usr/local/cula/lib64.

I was trying to run the following code from pgroup article http://www.pgroup.com/lit/articles/insider/v2n3a5.htm

Code: Select all
!  pgfortran -mp cula.cuf -L/usr/local/cula/lib64 -llapack -lcula_pgfortran -lblas -lcula
! export LD_LIBRARY_PATH=/usr/local/cula/lib64

        module cula_test
     
            use cudafor
            contains
                         
            ! gpu error reporting routine
            subroutine check_status(status)
           
                integer status
                integer info
                integer cula_geterrorinfo
                info = cula_geterrorinfo()
                if (status .ne. 0) then
                    if (status .eq. 7) then
                        write(*,*) 'invalid value for parameter ', info
                    else if (status .eq. 8) then
                        write(*,*) 'data error (', info ,')'
                    else if (status .eq. 9) then
                        write(*,*) 'blas error (', info ,')'
                    else if (status .eq. 10) then
                        write(*,*) 'runtime error (', info ,')'
                    else
                        call cula_getstatusstring(status)
                    endif
                    stop 1
                end if
               
            end subroutine
           
            ! cpu test (baseline)
            subroutine do_cpu_test(n,nrhs,ain,bin)
               
                ! input
                real,dimension(:,:) :: ain,bin
               
                ! allocations
                real,dimension(:,:),allocatable :: a,b,ans
                integer,dimension(:),allocatable :: ipiv
                integer n,nrhs
                integer c1,c2,cr,cm
                real norm
               
                ! back up input for reconstruction test
                allocate( a(n,n), b(n,nrhs), ipiv(n), ans(n,nrhs) )
                a = ain
                b = bin               
               
                ! start test
                call system_clock( c1, cr, cm )
                print *, 'starting cpu test...'
                               
                ! call lapack solver
                call sgesv(n,nrhs,a,n,ipiv,b,n,info)
               
                ! stop test
                call system_clock( count=c2 )
                print *, '  runtime:', 1.e3*real(c2-c1) / real(cr), 'ms'
                print *, '  gflops:', (0.66*n**3.) / (real(c2-c1) / real(cr)) / (1.e9)
               
                ! check answer
                ans = bin;
                call sgemm('n','n',n,nrhs,n,1.0,ain,n,b,n,-1.0,ans,n)
                norm = slange('1',n,nrhs,ans,n,work) / real(n)
                print *, '  error:', norm
                print *, ''
               
                ! cleanup
                deallocate(a,b,ipiv,ans)
               
            end subroutine do_cpu_test
           
            ! cula test (host interface)
            subroutine do_cula_host_test(n,nrhs,ain,bin)
               
                ! input
                real,dimension(:,:) :: ain,bin
               
                ! allocations (all on host)
                real,dimension(:,:),allocatable :: a,b,ans
                integer,dimension(:),allocatable :: ipiv
                integer n,nrhs,status
                integer c1,c2,cr,cm
                real norm
               
                ! back up input for reconstruction test
                allocate( a(n,n), b(n,nrhs), ipiv(n), ans(n,nrhs) )
                a = ain
                b = bin               
               
                ! start test
                call system_clock( c1,cr,cm )
                print *, 'starting cula (host interface) test...'
                               
                ! call cula solver (host interface)
                status = cula_sgesv(n,nrhs,a,n,ipiv,b,n)
                call check_status(status)
               
                ! stop test
                call system_clock( count=c2 )
                print *, '  runtime:', 1.e3*real(c2-c1) / real(cr), 'ms'
                print *, '  gflops:', (0.66*n**3.) / (real(c2-c1) / real(cr)) / (1.e9)
               
                ! check answer
                ans = bin;
                call sgemm('n','n',n,nrhs,n,1.0,ain,n,b,n,-1.0,ans,n)
                norm = slange('1',n,nrhs,ans,n,work) / real(n)
                print *, '  error:', norm
                print *, ''
               
                ! cleanup
                deallocate(a,b,ipiv,ans)
               
            end subroutine do_cula_host_test
                       
            ! cula test (device interface)
            subroutine do_cula_device_test(n,nrhs,ain,bin)
           
                ! input
                real,dimension(:,:) :: ain,bin
               
                ! allocations (all on host)
                real,dimension(:,:),allocatable :: a,b,ans
                integer n,nrhs,status
                integer c1,c2,cr,cm
                real norm
               
                ! gpu memory
                real,device,dimension(:,:),allocatable :: a_dev,b_dev
                integer,device,dimension(:),allocatable :: ipiv_dev
               
                ! back up input for reconstruction test
                allocate( a(n,n), b(n,nrhs), ans(n,nrhs) )
                a(1:n,1:n) = ain
                b(1:n,1:nrhs) = bin               
               
                ! allocate gpu memory
                allocate( a_dev(n,n), b_dev(n,nrhs), ipiv_dev(n) )
               
                ! start test
                call system_clock( c1,cr,cm )
                print *, 'starting cula (device interface) test...'
               
                ! copy memory to gpu
                a_dev = a
                b_dev = b
               
                ! call cula solver (device interface)
                status = cula_device_sgesv(n,nrhs,a_dev,n,ipiv_dev,b_dev,n)
               
                ! copy answer to host
                b = b_dev
               
                ! stop test
                call system_clock( count=c2 )
                print *, '  runtime:', 1.e3*real(c2-c1) / real(cr), 'ms'
                print *, '  gflops:', (0.66*n**3.) / (real(c2-c1) / real(cr)) / (1.e9)
               
                ! check answer
                ans(1:n,1:nrhs) = bin;
                call sgemm('n','n',n,nrhs,n,1.,ain,n,b,n,-1.,ans,n)
                norm = slange('1',n,nrhs,ans,n,work) / real(n)
                print *, '  error:', norm
                print *, ''
               
                ! cleanup
                deallocate(a,b,ans)
                deallocate(a_dev,b_dev,ipiv_dev)
               
            end subroutine do_cula_device_test
           
        end module cula_test
       
        ! main program

        program cula

            use cula_test
                   
            real error,eps
           
            ! Host memory
            real,dimension(:,:),allocatable :: a, b
            integer n, info, i, j, status

            n = 30
            nrhs = 1

            print *,'cula + pgfortran test (matrix solve)'
            print *,'  array size: ', n, ' by ', n
            print *,'  right hand sides: ', nrhs
            print *,''
            allocate( a(n,n), b(n,nrhs) )
                                   
            ! intialize a and b
            call random_number(a)
            call random_number(b)
            print *, b
            ! Make sure a() isn't singular
            do i=1,n
                a(i,i) = 10. * a(i,i) + 10.
            enddo
                       
            ! initialize cula
            status = cula_initialize()
            call check_status(status)
           
            ! do cpu test (baseline)
            call do_cpu_test(n,nrhs,a,b)
            print *, b                               
            ! do gpu test (host interface)
            call do_cula_host_test(n,nrhs,a,b)
            print *, b
            ! do gpu test (device interface)
            call do_cula_device_test(n,nrhs,a,b)
             
        end program cula


I also tried the gesv example from the CULA examples and it runs without any errors. So, its the code.
The code compiles fine, but I get a runtime error "Warning: ieee_inexact is signaling".

If I compile without the "stop" in line 25 and declare the functions cula_initialize as external, I get the CUDA runtime or driver not supported error.
hcula8
 
Posts: 5
Joined: Fri Aug 19, 2011 9:15 am

Re: CUDA runtime or driver is not supported

Postby john » Thu Aug 25, 2011 1:01 pm

This looks clean; what is the result of running ldd on the compiled executable?
john
Administrator
 
Posts: 587
Joined: Thu Jul 23, 2009 2:31 pm

Re: CUDA runtime or driver is not supported

Postby hcula8 » Thu Aug 25, 2011 1:59 pm

linux-vdso.so.1 => (0x00007fffc9370000)
libcula_pgfortran.so => /usr/local/cula/lib64/libcula_pgfortran.so (0x00007fcfe8437000)
libcudart.so.3 => /opt/pgi/linux86-64/2011/cuda/3.1/lib64/libcudart.so.3 (0x00007fcfe81fb000)
libpthread.so.0 => /lib/libpthread.so.0 (0x00007fcfe7fd0000)
librt.so.1 => /lib/librt.so.1 (0x00007fcfe7dc8000)
libm.so.6 => /lib/libm.so.6 (0x00007fcfe7b45000)
libc.so.6 => /lib/libc.so.6 (0x00007fcfe77c1000)
libstdc++.so.6 => /usr/lib/libstdc++.so.6 (0x00007fcfe74ad000)
libcula.so => /usr/local/cula/lib64/libcula.so (0x00007fcfe5a1e000)
libcublas.so.3 => /opt/pgi/linux86-64/2011/cuda/3.1/lib64/libcublas.so.3 (0x00007fcfe298b000)
libgomp.so.1 => /usr/lib/libgomp.so.1 (0x00007fcfe277d000)
libgcc_s.so.1 => /lib/libgcc_s.so.1 (0x00007fcfe2566000)
libdl.so.2 => /lib/libdl.so.2 (0x00007fcfe2361000)
/lib64/ld-linux-x86-64.so.2 (0x00007fcfe8645000)
hcula8
 
Posts: 5
Joined: Fri Aug 19, 2011 9:15 am

Re: CUDA runtime or driver is not supported

Postby john » Fri Aug 26, 2011 5:27 am

Code: Select all
libcudart.so.3 => /opt/pgi/linux86-64/2011/cuda/3.1/lib64/libcudart.so.3 (0x00007fcfe81fb000)

There it is, you're loading CUDA 3.1 due to how you have your PGI set up. See the section "PGI Accelerator and CUDA Fortran Enhancements" in your PGI docs. Each version of CULA lists the version of CUDA with which it is compatible, so you'll want to match the two. We would recommend CULA R12 and CUDA 4.0.
john
Administrator
 
Posts: 587
Joined: Thu Jul 23, 2009 2:31 pm

Re: CUDA runtime or driver is not supported

Postby hcula8 » Fri Aug 26, 2011 12:36 pm

Using the compiler flag -Mcuda=cuda3.2 solves the problem ! I appreciate your help.Thank you :D
hcula8
 
Posts: 5
Joined: Fri Aug 19, 2011 9:15 am


Return to Linux Support

Who is online

Users browsing this forum: No registered users and 1 guest

cron