Date: Mon, 13 Jun 2005 10:56:29 +0000 From: "Apache Xie" <apachexm@hotmail.com> To: freebsd-hackers@freebsd.org Subject: contigmalloc() and mmap() Message-ID: <BAY19-F33CED9E68F54579EE49ACCC4F00@phx.gbl>
next in thread | raw e-mail | index | archive | help
I have some experiences in writing Linux device driver, but I am new to FreeBSD kernel, although from the first glimpse, there seems no big differences between the kernel operations a char device driver can use, but I met some problems when the driver is running in FreeBSD. Our device is an experimental communication card, which can do remote DMA between two cards, which means the card in one node can DMA memory data to network, and when data are transfered to the card on another node, it will be DMAed to memory too. Because the card can only use contiguous physical memory for DMA operations, so data in user process will be copied to an contiguous memory buffer, then DMA will tranfer data in this buffer, and this buffer is allocated by driver using __get_free_pages() in Linux. The buffer is mmaped to user process space, so user process can do the copy directly in user space, it can directly orgnize data in this mmaped memory too. When I am porting my driver to FreeBSD, I use /dev/bktr driver as the example, seems easily, just using contigmalloc() to allocate the buffer, and in driver's _mmap() function, return the physical address for each page to be mmaped. The problem is, in Linux, I allocate buffer in driver's ioctl() function, and free it in a timer function, many processes may use the driver at the same time, each process use a different kernel buffer, when the process first use the driver, it calls __get_free_pages() to allocate kernel buffer, and when it exit, it trigger timer function, the timer function will can free_pages() to free the buffer, so these two kernel interfaces will be called frequently, but this usage pattern works correctly in Linux. In FreeBSD, the driver works in the same pattern, but when a user process mmap driver's buffer (allocated by contigmalloc()) and is killed, then when another process mmap the same buffer again, sometimes it cannot get correct data from the mmaped pages, which means the user space virtual aderess may not point to the correct physical page of driver's buffer, sometimes the OS even panic with some information such as "Trap 12, Page not present" etc. I browsed kernel tree, I found those drivers which use contigmalloc() and contigfree() always call these two kernel interfaces in _attach() and _detach(), but in my driver, I call contigmalloc() in ioctl(), and call contigfree() in a callout function which is set by callout_reset(). What I want to know is if contigmalloc() and contigfree() can only be used under some conditions? And recently, I modified my driver, I allocated a big chunk of contiguous physical memory using contigmalloc() in the driver's _attach() function, and I use a simple first-fit algorithm to manage this memory myself, which mean in ioctl() I use my allocate/deallocate functions instead of contigmalloc(), in _detach() function contigfree() is called to free the big chunk of memory, no panic again, but sometimes, process cannot get the correct data from the mmaped memory. I don't know why. Any help is welcomed. Thanks. _________________________________________________________________ Express yourself instantly with MSN Messenger! Download today it's FREE! http://messenger.msn.click-url.com/go/onm00200471ave/direct/01/
Want to link to this message? Use this URL: <https://mail-archive.FreeBSD.org/cgi/mid.cgi?BAY19-F33CED9E68F54579EE49ACCC4F00>