Location of Memory Allocation

General CULA Dense (LAPACK & BLAS) support and troubleshooting. Use this forum if you are having a general problem or have encountered a bug.

Location of Memory Allocation

Postby ssylee » Fri Nov 06, 2009 2:19 am

I have gone through some examples in the tools. I have noticed that the memory blocks for the matrices are allocated in the CPU memory rather than GPU memory. For performance purposes, does it matter where the memory is allocated? What would be the best practices in terms of using CULA in this context? Thanks in advance for the response.
ssylee
CULA Academic
 
Posts: 6
Joined: Thu Nov 05, 2009 11:08 am

Re:Location of Memory Allocation

Postby kyle » Fri Nov 06, 2009 9:01 am

Hi ssylee,

The quick answer is that CULA has two main interfaces; a host interface and a device interface. The host interface takes CPU memory pointers and the device interface takes GPU memory points. The CPU interface does all the memory transfers behind the scenes and also does some pitch management to increase memory bandwidth.

A lot more information regarding this subject can be found in this post:

http://www.culatools.com/forums/14-gene ... -interface

Or from our programmer's guide:

http://www.culatools.com/html_guide/#introduction
kyle
Administrator
 
Posts: 301
Joined: Fri Jun 12, 2009 7:47 pm

Re:Location of Memory Allocation

Postby kyle » Fri Nov 06, 2009 9:24 am

To answer the second part of your question, we'd consider the normal (host) interface to be a "best practice" unless you are planning on doing a series of GPU operations.

For larger matrices, the transfer time to and from the GPU accounts for less than 1% of the total runtime. Additionally, with the pitch management of the host interface, we have seen speed-ups of up to 50% for certain problems thanks to bandwidth optimizations.
kyle
Administrator
 
Posts: 301
Joined: Fri Jun 12, 2009 7:47 pm

Re:Location of Memory Allocation

Postby dan » Fri Nov 06, 2009 9:36 am

I'd also recommend comparing the "geqrf" and "device" examples. These show the differences between using the two interfaces.

Dan
dan
Administrator
 
Posts: 61
Joined: Thu Jul 23, 2009 2:29 pm

Re:Location of Memory Allocation

Postby ssylee » Mon Nov 09, 2009 2:42 am

Thanks for your advices on the location. However, I need your advices with respect to CULA for what I'm trying to do below:

1. Matrix_temp = B * dK, where both B and dK are matrices located in CUDA context memory, computed using cublas sgemm() function call.
2. compute the inverse of Matrix_temp using CULAtools Lapack functions (sgetrf and sgetri for starters, but I may end up using Cholesky factorization rather than LU factorization later on)
3. more series of sgemm multiplications using cublas, with matrices and results located in CUDA context memory

My question is if using the device interface would be better suited for what I'm trying to do here (in order to avoid unnecessary memory buffer transfers between the host and device). If I were to use the device interface, what would be the easiest way to interface between CULA device and CUDA device pointers? I've read the two recommended links and still couldn't quite figure it out, as nothing of what I asked is stated explicitly. Thanks in advance for your comments.
ssylee
CULA Academic
 
Posts: 6
Joined: Thu Nov 05, 2009 11:08 am

Re:Location of Memory Allocation

Postby kyle » Mon Nov 09, 2009 10:00 am

This seems like a good usage scenario for CULA's device interface. You are doing GPU operations before and after the CULA calls, so you can save a few memory transfers.

However, in this case you may have a significant drop in performance (~10-50%) if your data is poorly pitched on the GPU. This can be overcome by using the culaDeviceMalloc() and culaDeviceFree() memory management tools provided by CULA Premium.

For interfacing between CULA and CUDA pointers, there is nothing you need to do here. Both CULA and CUDA respect the built in 'float' and 'double' types. If you need some more assistance, check out the 'device' example or post some code you are having trouble compiling.
kyle
Administrator
 
Posts: 301
Joined: Fri Jun 12, 2009 7:47 pm


Return to CULA Dense Support

Who is online

Users browsing this forum: No registered users and 1 guest

cron